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Abstract: A serious crisis is identified in theories of neurocomputation, marked by a persistent disparity between the phenomenologi- 
cal or experiential account of visual perception and the neurophysiological level of description of the visual system. In particular, con- 
ventional concepts of neural processing offer no explanation for the holistic global aspects of perception identified by Gestalt theory. The 
problem is paradigmatic and can be traced to contemporary concepts of the functional role of the neural cell, known as the Neuron Doc- 
trine. In the absence of an alternative neurophysiologically plausible model, I propose a perceptual modeling approach, to model the 
percept as experienced subjectively, rather than modeling the objective neurophysiological state of the visual system that supposedly sub- 
serves that experience. A Gestalt Bubble model is presented to demonstrate how the elusive Gestalt principles of emergence, reifica- 
tion, and invariance can be expressed in a quantitative model of the subjective experience of visual consciousness. That model in turn 
reveals a unique computational strategy underlying visual processing, which is unlike any algorithm devised by man, and certainly un- 
like the atomistic feed-forward model of neurocomputation offered by the Neuron Doctrine paradigm. The perceptual modeling ap- 
proach reveals the primary function of perception as that of generating a fully spatial virtual-reality replica of the external world in an in- 
ternal representation. The common objections to this “picture-in-the-head” concept of perceptual representation are shown to be ill 


founded. 
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1. Introduction 


Contemporary neuroscience finds itself in a state of serious 
crisis, for the deeper we probe into the workings of the 
brain, the farther we seem to get from the ultimate goal of 
providing a neurophysiological account of the mechanism 
of conscious experience. Nowhere is this impasse more ev- 
ident than in the study of visual perception, where the ap- 
parently clear and promising trail discovered by Hubel and 
Wiesel (1959) leading up the hierarchy of feature detection 
from primary to secondary and to higher cortical areas 
seems to have reached a theoretical dead end. Besides the 
troublesome issues of the noisy stochastic nature of the 
neural signal and the very broad tuning of the single cell as 
a feature detector, the notion of visual processing as a hier- 
archy of feature detectors seems to suggest some kind of 
“grandmother cell” model in which the activation of a sin- 
gle cell or a group of cells represents the presence of a par- 
ticular type of object in the visual field. However, it is not 
at all clear how such a featural description of the visual 
scene could even be usefully employed in practical interac- 
tion with the world. 

Alternative paradigms of neural representation have 
been proposed, including the suggestion that synchronous 
oscillations play a role in perceptual representation, al- 
though these theories are not yet specified sufficiently to 
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know exactly how they address the issue of perceptual rep- 
resentation. But the most serious indictment of contempo- 
rary neurophysiological theories is that they offer no hint of 
an explanation for the subjective experience of visual con- 
sciousness. Visual experience is more than just an abstract 
recognition of the features present in the visual field — those 
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features are vividly experienced as solid three-dimensional 
objects, bounded by colored surfaces, embedded in a spatial 
void. A number of enigmatic properties of this world of ex- 
perience were identified decades ago by Gestalt theory, sug- 
gestive of a holistic emergent computational strategy whose 
operational principles remain a mystery. 

The problem in modern neuroscience is a paradigmatic 
one that can be traced to its central concept of neural pro- 
cessing. According to the Neuron Doctrine, neurons behave 
as quasi-independent processors separated by relatively 
slow chemical synapses, with strictly segregated input and 
output functions through the dendrites and axon, respec- 
tively. It is hard to imagine how such an assembly of inde- 
pendent processors could account for the holistic emergent 
properties of perception identified by Gestalt theory. In 
fact, the reason these Gestalt aspects of perception have 
been largely ignored in recent decades is exactly because 
they are so difficult to express in terms of the Neuron Doc- 
trine paradigm. More recent proposals that implicate syn- 
chronous oscillations as the neurophysiological basis of con- 
scious experience (Crick 1994; Crick & Koch 1990; Eckhorn 
et al. 1988; Llinas et al. 1994; Singer 1999; Singer & Gray 
1995) seem to suggest some kind of holistic global process 
that appears to be more consistent with Gestalt principles, 
although it is hard to see how this paradigm, at least as cur- 
rently conceived, can account for the solid three-dimen- 
sional nature of subjective experience. The persistent dis- 
parity between the neurophysiological and phenomenal 
levels of description suggests that either the subjective ex- 
perience of visual consciousness is somehow illusory, or the 
state of our understanding of neural representation is far 
more embryonic than is generally recognized. 

Pessoa et al. (1998) made the case for denying the pri- 
macy of conscious experience. They argued that although 
the subjective experience of filling-in phenomena is some- 
times accompanied by a neurophysiological correlate, such 
an isomorphism between experience and neurophysiology 
is not logically necessary but is merely an empirical issue. 
For, they claimed, subjective experiences can occur in the 
absence of a strictly isomorphic correlate. Their view is that 
although the subjective experience of visual consciousness 
appears as a “picture” or three-dimensional model of a sur- 
rounding world, this does not mean that the information 
manifest in that experience is necessarily explicitly encoded 
in the brain. Moreover, that consciousness is an illusion 
based on a far more compressed or abbreviated represen- 
tation, in which percepts such as that of a filled-in colored 
surface can be explained neurophysiologically by “ignoring 
an absence” rather than by an explicit point-for-point map- 
ping of the perceived surface in the brain. 

In fact, nothing could be further from the truth. For to 
propose that the subjective experience of perception can be 
more enriched and explicit than the corresponding neuro- 
physiological state, flies in the face of the materialistic basis 
of modern neuroscience. The modern view is that mind and 
brain are different aspects of the same physical mechanism. 
In other words, every perceptual experience, whether a sim- 
ple percept such as a filled-in surface or a complex percept 
ofa whole scene, has two essential aspects, the subjective ex- 
perience of the percept and the objective neurophysiologi- 
cal state of the brain that is responsible for that subjective 
experience. Like the two faces of a coin, these very different 
entities can be identified as merely different manifestations 
of the same underlying structure, viewed from the internal 


376 BEHAVIORAL AND BRAIN SCIENCES (2003) 26:4 


first-person perspective as opposed to the external third- 
person perspective. The dual nature of a percept is analo- 
gous to the representation of data in a digital computer, 
where a pattern of voltages present in a particular memory 
register can represent some meaningful information, such 
as a numerical value, a brightness value in an image, or a 
character of text, when viewed from inside the appropriate 
software environment, but when viewed in external physical 
terms those same data take the form of voltages or currents 
in particular parts of the machine. However, whatever form 
is selected for encoding data in the computer, the informa- 
tion content of that data cannot possibly be of higher di- 
mensionality than the information explicitly expressed in the 
physical state of the machine. 

The same principle must also hold in perceptual experi- 
ence, as proposed by Miiller (1896) in the psychophysical 
postulate. Miiller argued that because the subjective expe- 
rience of perception is encoded in some neurophysiologi- 
cal state, the information encoded in that conscious experi- 
ence cannot possibly be any greater than the information 
encoded in the corresponding neurophysiological state. Al- 
though we cannot observe phenomenologically the physi- 
cal medium by which perceptual information is encoded in 
the brain, we can observe the information encoded in that 
medium, expressed in terms of the variables of subjective 
experience. It follows therefore that it should be possible 
by direct phenomenological observation to determine the 
dimensions of conscious experience, and thereby to infer 
the dimensions of the information encoded neurophysio- 
logically in the brain. 

The bottom-up approach that works upward from the 
properties of the individual neuron and the top-down ap- 
proach that works downward from the subjective experi- 
ence of perception are equally valid and complementary 
approaches to the investigation of the visual mechanism. 
Eventually, these opposite approaches to the problem must 
meet somewhere in the middle. To date, however, the gap 
between them remains as large as it ever was. Both ap- 
proaches are essential to the investigation of biological vi- 
sion because each offers its own unique perspective on the 
problem. The disparity between these two views of the vi- 
sual representation helps to maintain the focus on exactly 
those properties that are prominently absent from the con- 
ventional neural network view of visual processing. 


2. The epistemological divide 


There is a central philosophical issue that underlies discus- 
sions of phenomenal experience as seen, for example, in the 
distinction between the Gestaltist and the Gibsonian views 
of perception. That is, the epistemological question of 
whether the world we see around us is the real world itself 
or merely an internal perceptual copy of that world gener- 
ated by neural processes in our brain. In other words, this 
is the question of direct realism (also known as naive real- 
ism) as opposed to indirect realism (or representational- 
ism). To take a concrete example, consider the vivid spatial 
experience of this paper that you hold in your hands. The 
question is whether the rich spatial structure of this experi- 
ence before you is the physical paper itself, or an internal 
data structure or pattern of activation within your physical 
brain. Although this issue is not much discussed in con- 
temporary psychology, it is an old debate that has resur- 
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faced several times in psychology, and the failure to reach 
consensus on this issue continues to bedevil the debate on 
the functional role of sensory processing. The reason for the 
continued confusion is that both direct and indirect realism 
are frankly incredible, although each is incredible for dif- 
ferent reasons. 


2.1. Problems with direct realism 


The direct realist view is incredible because it suggests that 
we can have experience of objects out in the world directly, 
beyond the sensory surface, as if bypassing the chain of sen- 
sory processing. For example, if light from this paper is 
transduced by your retina into a neural signal that is trans- 
mitted from your eye to your brain, then the very first as- 
pect of the paper that you can possibly experience is the 
information at the retinal surface, or the perceptual repre- 
sentation that is downstream of it in your brain. The physi- 
cal paper itself lies beyond the sensory surface and there- 
fore must be beyond your direct experience. But the 
perceptual experience of the page stubbornly appears out 
in the world itself instead of in your brain, in apparent vio- 
lation of everything we know about the causal chain of vi- 
sion. Gibson explicitly defended the notion of direct per- 
ception and spoke as if perceptual processing occurs 
somehow out in the world itself rather than as a computa- 
tion in the brain based on sensory input (Gibson 1972, 
pp. 217, 239). 

Significantly, Gibson refused to discuss sensory process- 
ing at all and even denied that the retina records anything 
like a visual image that is sent to the brain. This leaves the 
status of the sensory organs in a peculiar kind of limbo, for 
if the brain does not process sensory input to produce an in- 
ternal image of the world, what is the purpose of all that 
computational wetware? Another embarrassment for direct 
perception is the phenomenon of visual illusions, which are 
observed out in the world itself; and yet they cannot possi- 
bly be in the world for they are the result of perceptual pro- 
cessing that must occur within the brain. With characteris- 
tic aplomb, Gibson simply denied that illusions are illusory 
at all, although it is not clear exactly what he could possibly 
have meant by that. Modern proponents of Gibson’s theo- 
ries usually take care to disclaim his most radical views 
(Bruce & Green 1987, pp. 190, 203-204; O’Regan 1992, 
p. 473; Pessoa et al. 1998), but they present no viable alter- 
native explanation to account for our experience of the 
world beyond the sensory surface. 

The difficulty with the concept of direct perception is 
most clearly seen when we consider how an artificial vision 
system could be endowed with such external perception. 
Although a sensor may record an external quantity in an in- 
ternal register or variable in a computer, from the internal 
perspective of the software running on that computer, only 
the internal value of that variable can be “seen” or can pos- 
sibly influence the operation of that software. In an exactly 
analogous manner the pattern of electrochemical activity 
that corresponds to our conscious experience can take a 
form that reflects the properties of external objects, but our 
consciousness is necessarily confined to the experience of 
those internal effigies of external objects, rather than of the 
external objects themselves. Unless the principle of direct 
perception can be demonstrated in a simple artificial sen- 
sory system, this explanation remains as mysterious as the 
property of consciousness it is supposed to explain. 


2.2. Problems with indirect realism 


The indirect realist view is also incredible, for it suggests 
that the solid stable structure of the world we perceive to 
surround us is merely a pattern of energy in the physical 
brain; that is, the world that appears to be external to our 
head is actually inside our head. This could only mean that 
the head we have come to know as our own is not our true 
physical head but is merely a miniature perceptual copy of 
our head inside a perceptual copy of the world, all of which 
is completely contained within our true physical skull. 
Stated from the internal phenomenal perspective, out be- 
yond the farthest things you can perceive in all directions 
(ie., above the dome of the sky and below the earth under 
your feet, or beyond the walls, floor, and ceiling of the room 
you perceive around you), beyond those perceived surfaces 
is the inner surface of your true physical skull encompass- 
ing all that you perceive, and beyond that skull is an 
unimaginably immense external world, of which the world 
you see around you is merely a miniature virtual-reality 
replica. The external world and its phenomenal replica can- 
not be spatially superimposed, for one is inside your physi- 
cal head and the other is outside. Therefore, the vivid spa- 
tial structure of this page that you perceive here in your 
hands is itself a pattern of activation within your physical 
brain, and the real paper of which it is a copy is out beyond 
your direct experience. 

I have found a curious dichotomy in the responses of col- 
leagues in discussions on this issue. Many people agree with 
the statement that everything you perceive is in some sense 
inside your head, and in fact they often complain that this 
is so obvious it need hardly be stated. However, when that 
statement is turned around to say that out beyond every- 
thing you perceive is your physical skull, they object most 
vehemently that that is absurd. And yet the two statements 
are logically identical, so how can one appear trivially obvi- 
ous while the other seems patently absurd? The value of 
this particular mental image is that it helps to smoke out any 
residual naive realism that may remain hidden in our phi- 
losophy. For although this statement can only be true in a 
topological, rather than a strict topographical, sense, this in- 
sight emphasizes the indisputable fact that no aspect of the 
external world can possibly appear in consciousness except 
by being represented explicitly in the brain. The existential 
vertigo occasioned by this mental image is so disorienting 
that only a handful of researchers have seriously enter- 
tained this notion or pursued its implications to its logical 
conclusion (Harrison 1989; Hoffman 1998; Kant 1781/ 
1991; Koffka 1935; Kéhler 1971, p. 125; Lehar 2003b; Rus- 
sell 1927, pp. 137-143; Smythies 1989; 1994). 

Another reason the indirect realist view is incredible is 
that the observed properties of the world of experience 
when viewed from the indirect realist perspective are diffi- 
cult to resolve with contemporary concepts of neurocom- 
putation. For the world we perceive around us appears as a 
solid spatial structure that maintains its structural integrity 
as we turn around and move about in the world. Perceived 
objects within that world maintain their structural integrity 
and recognized identity as they rotate, translate, and scale 
by perspective in their motions through the world. These 
properties of the conscious experience fly in the face of 
everything we know about neurophysiology, for they sug- 
gest some kind of three-dimensional imaging mechanism in 
the brain, capable of generating three-dimensional volu- 
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metric percepts of the degree of detail and complexity ob- 
served in the world around us. No plausible mechanism has 
ever been identified neurophysiologically which exhibits 
this incredible property. The properties of the phenomenal 
world are therefore inconsistent with contemporary con- 
cepts of neural processing, which is exactly why these prop- 
erties have been so long ignored. 


2.3. Spirituality, supervenience, and other nomological 
danglers 


The perceived incredibility of both direct and indirect re- 
alism has led many over the centuries to propose that con- 
scious experience is located neither in the physical brain 
nor in the external world, but in some separate space that 
bears no spatial relation to the physical space known to sci- 
ence. These theories fall somewhere between direct and in- 
direct perception because they claim that phenomenal ex- 
perience is neither in the head, nor out in the world. The 
original formulation of this thesis was Cartesian dualism, 
the traditional religious or spiritual view that mind exists in 
a separate realm that is inaccessible to science. Our inabil- 
ity to detect spiritual entities is not due to any limitations of 
our detector technology but to the fact that spiritual enti- 
ties are impossible in principle to detect by physical means. 
Cartesian dualism is a minority position in contemporary 
philosophy, at least as a scientific theory of mind, and for 
very good reason. The chief objection to this kind of dual- 
ism is Occam’s razor: It is more parsimonious to posit a sin- 
gle universe with one set of physical laws rather than two 
radically dissimilar parallel universes composed of dissimi- 
lar substances and following dissimilar laws, making tenu- 
ous contact with each other nowhere else but within a liv- 
ing conscious brain. But if mind and matter come into 
causal contact, as they clearly do in both sensory and motor 
function, then surely they must be different parts of one 
and the same physical universe. There is another, still more 
serious objection to Cartesian dualism than the issue of par- 
simony. Since the experiential, or spiritual component of 
the theory is in principle inaccessible to science, that por- 
tion of the theory can be neither confirmed nor refuted. 
This places the spiritual component of Cartesian dualism 
beyond the bounds of science and firmly in the realm of re- 
ligious belief. 

A more sophisticated halfway epistemology is seen in the 
philosophy of critical realism (Broad 1925; Drake et al. 
1920: Russell 1921; Sellars 1916). Critical realists avoid re- 
ligious explanations involving God or spirits, but their con- 
cept of conscious experience nevertheless preserves some 
of the mystery of Cartesian dualism. Critical realists ac- 
knowledge that perception is not direct, but instead, is me- 
diated by an intermediate representational entity called 
sense-data. However, critical realists insist that sense-data 
are 

particular existents of a peculiar kind; they are not physical, .. . 

and there is no reason to suppose that they are either states of 

mind or existentially mind-dependent. In having spatial char- 
acteristics ... they resemble physical objects ... but in their 
privacy and their dependence on the body .. . of the observer 

they are more like mental states. (Broad 1925, p. 181) 


As with the spirit world of the Cartesian view, sense data 
and the space in which they are observed are not just diffi- 
cult to detect, but they are in principle beyond scientific 
scrutiny. 
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There is some debate among critical realists over the on- 
tology of conscious experience. In a book on critical realism 
by a consortium of authors (Drake et al. 1920), Lovejoy, 
Pratt, and Sellars claimed that the sensa are completely “the 
character of the mental existent ... although its existence 
is not given” (pp. 20-21), while Drake, Rogers, Santayana, 
and Strong agreed that the data are characteristic of the ap- 
prehended object, although “the datum is, gua datum, a 
mere essence, an inputed but not necessarily actual exis- 
tent. It may or may not have existence” (Drake 1920 in 
Drake et al. 1920, pp. 20-21, footnote). So the critical re- 
alists solved the epistemological problem by defining a 
unique kind of existent that is experienced, but that does 
not or may not actually exist. This is a peculiar inversion of 
the true epistemological situation because, in fact, sense 
data, or the raw material of conscious experience, are the 
only thing we can know with any real certainty to actually 
exist. All else, including the entire physical world known to 
science, is informed conjecture based on that experience. 

A more modern reformulation of this muddled episte- 
mology is seen in Davidson’s (1970) anomalous monist the- 
sis. Davidson suggested that the mental domain, on the ba- 
sis of its essential anomalousness and normativity, cannot be 
the object of serious scientific investigation because the 
mental is on a wholly different plane from the physical. This 
argument sounds like the metaphysical dualism of 
Descartes which disconnects mind from brain entirely, ex- 
cept that Davidson qualified his theory with the monistic 
proviso that every mental event is connected with specific 
physical events (in the brain), although there are no laws 
connecting mental kinds with physical kinds, and this pre- 
sumably rescues the thesis from metaphysical dualism. Kim 
(1998) pointed out, however, that this is a negative thesis, 
for it tells us only how the mental is not related to the phys- 
ical, it says nothing about how they are related. As such, this 
is more an article of faith rather than a real theory of any 
sort, and in the context of the history of the epistemologi- 
cal debate this can be seen as a last desperate attempt to 
rescue naive realism from its own logical contradictions. 
This kind of physicalism has been appropriately dubbed 
“token physicalism,” for it is indeed a token admission of the 
undeniable link between mind and the physical brain, with- 
out admitting to any of its very significant implications. 

To rationalize this view of the mind-brain relation, 
Davidson (1970) introduced the peculiar notion of super- 
venience, a one-way asymmetrical relation between mind 
and brain which makes the mind dependent on the brain 
but forever closes the possibility of phenomenological ob- 
servation of brain states. As in the case of Cartesian dual- 
ism, there are two key objections to this argument. In the 
first place, the disconnection between the experiential 
mind and the physical brain is itself merely a hypothesis 
whose truth remains to be demonstrated. It is at least 
equally likely prima facie that the mind does not supervene 
on the brain, but rather that the mind is identically equal to 
the functioning of the physical brain. In fact, this is by far 
the more parsimonious explanation because it invokes a sin- 
gle explanans, the physical brain, to account for the prop- 
erties of both mind and brain. After all, physical damage to 
the brain can result in profound changes in the mind, not 
just in the information content of the mind or in observed 
behavior but in the experiential or “what it is like” aspect of 
conscious experience. The simplest explanation therefore is 
that consciousness is a physical process taking place in the 


Lehar: Gestalt isomorphism and the primacy of subjective conscious experience 


physical brain, which is why it is altered by physical changes 
to the physical brain. 

But the problem of supervenience is more serious than 
just the argument of parsimony. If the properties of mind 
are indeed disconnected from the properties of the physi- 
cal brain, this would leave the mental domain completely 
disconnected from the world of reality known to science, as 
what Feig] (1958) has called a “nomological dangler.” If the 
properties of mind are not determined by the properties of 
the physical brain, what is it that determines the properties 
of the mind? For example, phenomenal color experience 
has been shown to be reducible to the three dimensions of 
hue, intensity, and saturation. Physical light is not restricted 
to these three dimensions; the spectrum of a typical sample 
of colored light contains a separate and distinct magnitude 
for every spectral frequency of the light, an essentially infi- 
nite-dimensional space that is immeasurably greater in in- 
formation content than the three dimensions of phenome- 
nal color experience. In answer to Koffka’s (1935) classical 
question “Why do things look as they do?”, the answer is 
clearly not “Because they are what they are.” That answer 
is clearly false in the case of color perception, as well as in 
the cases of visual illusions, dreams, and hallucinations. We 
now know that the dimensionality of color experience re- 
lates directly to the physiology of color vision; it relates to 
the fact that there are three different cone types in the hu- 
man retina and it relates to the opponent color process rep- 
resentation in the visual cortex. The dimensions of color ex- 
perience therefore are not totally disconnected from the 
properties of the physical brain, as suggested by Davidson 
(1970), but in fact phenomenal color experience tells us 
something very specific about the properties of the repre- 
sentation of color in the physical brain. And the same argu- 
ment holds for spatial vision, for there are a number of 
prominent distortions of phenomenal space which clearly 
indicate that phenomenal space is ontologically distinct 
from the physical space known to science, as will be dis- 
cussed in section 6.3. 

Daniel Dennett (1991) promoted a similar halfway epis- 
temology by drawing a distinction between the neural ve- 
hicles of mental representation and the phenomenal con- 
tents of those vehicles. Dennett opened the epistemological 
crack by claiming that the phenomenal contents do not nec- 
essarily bear any similarity whatsoever to the neural vehi- 
cles by which they are encoded in the brain. This actually 
goes beyond Davidson’s supervenience because, according 
to Davidson (1970), mental events that are distinct phe- 
nomenally must also be distinct neurophysiologically. This 
is tantamount to saying that the dimensions of conscious ex- 
perience cannot be any less than the dimensions of the cor- 
responding neurophysiological state. Dennett effectively 
removed this limitation by suggesting that even the dimen- 
sionality of the phenomenal contents need not match that 
of the neural vehicles. And into that epistemological crack, 
Dennett slipped the entire world of conscious experience 
like a magical disappearing act, where it is experienced but 
does not actually exist. By the very fact that conscious ex- 
perience, as conceived by Dennett, is in principle unde- 
tectable by scientific means, this concept of consciousness 
becomes a religious rather than a scientific hypothesis, 
whose existence can be neither confirmed nor refuted by 
scientific means. In fact, Dennett even suggested that there 
is actually no such thing as consciousness per se, and that 
belief in consciousness is akin to belief in some kind of 


mythical nonexistent deity (Dennett 1981). This argument 
of course is only intelligible from a naive realist perspective, 
by which the sense-data of conscious experience, so plainly 
manifest to one and all, are misidentified as the external 
world itself rather than as something going on in the phys- 
ical brain. 

Another modern theorist, Max Velmans (1990), revived 
an ancient notion of perception as something projecting out 
of the head into the world, as proposed by Empedocles and 
promoted by Malebranche. But Velmans refined this an- 
cient notion with the critical realist proviso that nothing 
physical actually gets projected from the head; the only 
thing that is projected is conscious experience, a subjective 
quality that is undetectable externally by scientific means. 
But again, as with critical realism, the problem with this no- 
tion is that the sense-data that are experienced to exist do 
not exist in any true physical sense, and therefore the pro- 
jected entity in Velman’s theory is a spiritual entity to be be- 
lieved in (for those who are so inclined), rather than any- 
thing knowable by, or demonstrable to, science. Velmans 
drew the analogy of a videotape recording that carries the 
information of a dynamic pictorial scene, expressed in a 
highly compressed and nonspatial representation, as pat- 
terns of magnetic fields on the tape. There is no resem- 
blance or isomorphism between the magnetic tape and the 
images that it encodes, except for its information content. 
However, the only reason the videotape even represents a 
visual scene is because of the existence of a video technol- 
ogy that is capable of reading the magnetic information 
from the tape and sweeping it out as a spatial image on a 
video monitor or television screen, where each pixel ap- 
pears in its proper place in the image. If that equipment did 
not exist, there would be no images as such on the video- 
tape. But if video technology is to serve as an analogy for 
spatial representation in the brain, the key question is 
whether the brain encodes that pictorial information exclu- 
sively in abstract compressed form like the magnetic pat- 
terns on the tape, or whether the brain reads those com- 
pressed signals and projects them as an actual spatial image 
somewhere in the brain like a television monitor, whenever 
we have a visuospatial experience. If it is the former, then 
sense-data are experienced but do not actually exist as a sci- 
entific entity, so the spatial image we see is a complete illu- 
sion, which, again, is an inversion of the true epistemology. 
If itis the latter, then there are actual “pictures in the head,” 
a notion that Velmans emphatically rejected. 

In fact, the only epistemology that is consistent with the 
modern materialistic world view is an identity theory (Feig] 
1958; Russell 1927) whereby mind is identically equal to 
physical patterns of energy in the physical brain. To claim 
otherwise is to relegate the elaborate structure of conscious 
experience to a mystical state beyond the bounds of science. 
The dimensions of conscious experience, such as phenom- 
enal color and phenomenal space, are a direct manifesta- 
tion of certain physical states of our physical brain. The only 
right answer to Koffka’s question (Koffka 1935) is that 
things appear as they do because that is the way the world 
is represented in the neurophysiological mechanism of our 
physical brain. In principle, therefore, the world of con- 
scious experience is accessible to scientific scrutiny after all, 
both internally through introspection and_ externally 
through neurophysiological recording. And introspection is 
as valid a method of investigation as is neurophysiology, just 
as in the case of color experience. Of course, the mind can 
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be expected to appear quite different from these two per- 
spectives, just as the data in a computer memory chip ap- 
pear quite different when examined internally by data ac- 
cess as opposed to externally by electrical probes. But the 
one quantity that is preserved across the mind/brain barrier 
is information content, and therefore that quantity can help 
to identify the neurophysiological mechanism or principle 
in the brain whose dimensionality, or information content, 
matches the observed dimensions of conscious experience. 


2.4. Selection from incredible alternatives 


We are left therefore with three alternatives, each of which 
appears to be absolutely incredible. Contemporary neuro- 
science seems to take something of an equivocal position on 
this issue, recognizing the epistemological limitations of the 
direct realist view and of the projection hypothesis, yet be- 
ing unable to account for the incredible properties sug- 
gested by the indirect realist view. However, one of these 
three alternatives simply must be true, to the exclusion of 
the other two. And the issue is by no means inconsequen- 
tial, for these opposing views suggest very different ideas of 
the function of visual processing, or what all that neural 
wetware is supposed to actually do. Therefore, it is of cen- 
tral importance for psychology to address this issue head- 
on, and to determine which of these competing hypotheses 
reflects the truth of visual processing. Until this most cen- 
tral issue is resolved definitively, psychology is condemned 
to remain in what Kuhn (1970) calls a pre-paradigmatic 
state, with different camps arguing at cross-purposes due to 
a lack of consensus on the foundational assumptions and 
methodologies of the science. Psychology is, after all, the 
science of the psyche, the subjective side of the mind/brain 
barrier, and neurophysiology only enters the picture to pro- 
vide a physical substrate for mind. Therefore, it is of vital 
importance to reach a consensus on the nature of the ex- 
planandum of psychology before we can attempt an ex- 
planans. In particular, we must decide whether the vivid 
spatial structure of the surrounding world of visual experi- 
ence is an integral part of the psyche and thus within the ex- 
planandum of psychology, or whether it is the external 
world itself, as it appears to be naively, and thus in the 
province of physics rather than of psychology. 

The problem with the direct realist view is of an episte- 
mological nature, and is therefore a more fundamental ob- 
jection; for direct realism, as defended by Gibson (1979), is 
nothing short of magical — that we can see the world out be- 
yond the sensory surface. The projection theory has a sim- 
ilar epistemological problem and is equally magical and 
mysterious, suggesting as it does that neural processes in 
our brain are somehow also out in the world. Both of these 
paradigms have difficulty with the phenomena of dreams 
and hallucinations (Revonsuo 1995), which present the 
same kind of phenomenal experience as spatial vision, ex- 
cept independent of the external world in which that per- 
ception is supposed to occur in normal vision. It is the im- 
plicit or explicit acceptance of this naive concept of 
perception which has led many to conclude that conscious- 
ness is deeply mysterious and forever beyond human com- 
prehension. For example, Searle (1992, p. 96) contended 
that consciousness is impossible to observe, for when we at- 
tempt to observe consciousness we see nothing but what- 
ever it is that we are conscious of; there is no distinction be- 
tween the observation and the thing observed. 
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On the other hand, the problem with the indirect realist 
view is more of a technological or computational limitation, 
for we cannot imagine how contemporary concepts of neu- 
rocomputation, or even of artificial computation for that 
matter, can account for the properties of perception as ob- 
served in visual consciousness. It is clear, however, that the 
most fundamental principles of neural computation and 
representation remain to be discovered, and therefore we 
cannot allow our currently limited notions of neurocompu- 
tation to constrain our observations of the nature of visual 
consciousness. The phenomena of dreams and hallucina- 
tions clearly demonstrate that the brain is capable of gen- 
erating vivid spatial percepts of a surrounding world inde- 
pendent of that external world, and that capacity must be a 
property of the physical mechanism of the brain. Normal 
conscious perception can therefore be characterized as a 
guided hallucination (Revonsuo 1995), which is as much a 
matter of active construction as it is of passive detection. If 
we accept the truth of indirect realism, this immediately 
disposes of at least one mysterious or miraculous compo- 
nent of consciousness, which is its unobservability. Con- 
sciousness is indeed observable, contrary to Searle’s con- 
tention, because the objects of experience are first and 
foremost the product or “output” of consciousness, and 
only in secondary fashion are they also representative of ob- 
jects in the external world. Searle’s (1992) difficulty in ob- 
serving consciousness is analogous to saying that you can- 
not see the moving patterns of glowing phosphor on your 
television screen, all you see is the ball game that is show- 
ing on that screen. The indirect realist view of television is 
that what you are seeing is first and foremost glowing phos- 
phor patterns on a glass screen, and only in secondary fash- 
ion are those moving images also representative of the re- 
mote ball game. 

The choice therefore is between accepting a magical 
mysterious account of perception and consciousness that 
seems impossible in principle to implement in any artificial 
vision system, or facing the seemingly incredible truth that 
the world we perceive around us is indeed an internal data 
structure within our physical brain (Lehar 2003b). The 
principal focus of neurophysiology should now be to iden- 
tify the operational principles behind the three-dimen- 
sional volumetric imaging mechanism in the brain, the 
mechanism responsible for generating the solid stable 
world of visual experience that we observe to surround us 
in conscious experience. 


3. Problems in modeling perception 


The computational modeling of perceptual processes is a 
formidable undertaking. But the problem is exacerbated by 
the fact that a neural network model of perception attempts 
to model two entities simultaneously: the subjective expe- 
rience of perception and the neurophysiological mecha- 
nism by which that experience is generated in the brain. 
The chief problem with this approach is that our knowledge 
of neurophysiological principles is known to be incomplete. 
We do not understand the computational functionality of 
even the simplest neural systems. For example, the lowly 
house fly, with its tiny pinpoint of a brain, seems to thumb 
its nose at our lofty algorithms and complex computational 
models as it dodges effortlessly between the tangled 
branches of a shrub in dappled sunlight, compensating for 
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gusty cross-winds to avoid colliding with the branches. This 
remarkable performance by this lowly creature far exceeds 
the performance of our most powerful computer algo- 
rithms and our most sophisticated neural network models 
of human perception. In fact, the “dirty little secret” of neu- 
roscience, as Searle (1997, p. 198) called it, is that we have 
no idea what the right level of analysis of the brain should 
be because there is no universally accepted theory of how 
the brain actually codes perceptual or experiential infor- 
mation. The epistemological question highlights this un- 
certainty, for it shows that there is not much consensus on 
whether the world of conscious experience is even explic- 
itly represented in the brain at all, the majority view being, 
apparently, that it is not. Palmer (1999) went even further, 
saying that “to this writer’s knowledge, no one has ever sug- 
gested any theory that the scientific community regards as 
giving even a remotely plausible causal account of how ex- 
perience arises from neural events.” Without this key piece 
of knowledge, how can we even begin to model the com- 
putational processes of perception in neurophysiological 
terms? 

One approach is to begin with the neurophysiology of the 
brain and attempt to discover what it is computing at the lo- 
cal level of the individual neuron, the elemental building 
block of the nervous system. The fruit of this branch of in- 
vestigation is neural network theory. But it is unclear 
whether neural network theory offers an adequate charac- 
terization of the actual processing going on in the brain, or 
whether it is asking too much of simple integrate-and-fire 
elements, no matter how cleverly connected in patterns of 
synaptic connections, to provide anything like an adequate 
account of the observed properties of conscious experience. 
Churchland (1984) argued in the affirmative, that we do 
have enough knowledge of the principles of neurocompu- 
tation to begin to propose realistic models of perceptual 
processing. Palmer (1992) and Opie (1999) presented dy- 
namic neural network models of Gestalt phenomena, such 
as the perceptual grouping of triangles, showing how the 
dynamics of perceptual phenomena can be modeled by a 
dynamic neural network model. But those models are pro- 
posed in the abstract, presenting general principles rather 
than complete and detailed models of specific perceptual 
phenomena expressed as sense-data. For example, Palmer 
(1992) discussed the perceptual experience of an equilat- 
eral triangle, perceived as an arrow pointing in one of three 
directions. Palmer modeled this perceptual phenomenon 
as a competition between three dynamic neural network 
nodes in a mutually inhibitory relationship, resulting in a 
“winner-take-all” behavior. Although this model is com- 
pelling as a demonstration of Gestalt principles in a neural 
network model, Palmer left out the most difficult part of the 
problem, which is not just the competition between three 
alternative percepts but the perceptual representation of 
the percept itself. The perceptual experience of a triangle 
cannot be reduced to just three phenomenal values but is 
observed as a fully reified triangular structure that spans a 
specific portion of perceived space. This sense-data com- 
ponent of the phenomenal experience is very much more 
difficult to account for in neural network terms. 

In recent decades a number of attempts have been made 
to quantify the sense-data of visual consciousness in com- 
putational models (see Lesher 1995, for a review). Zucker 
et al. (1988) presented a model of curve completion that ac- 
counts for the emergent nature of perceptual processing by 


incorporating a feedback loop in which local feature detec- 
tors tuned to detect oriented edges feed up to global cur- 
vature detector cells, and those cells in turn feed back down 
to the local edge level to fill in missing pieces of the global 
curve. A similar bottom-up/top-down feedback is given in 
Grossberg and Mingolla’s (1985) visual model to account 
for boundary completion in illusory figures like the Kanizsa 
square by generating an explicit line of neural activation 
along the illusory contour. An extension of that model 
(Grossberg & Todorovicz 1988) accounted for the filling-in 
of the surface brightness percept in the Kanizsa figure, with 
an explicit diffusion of neural activation within the region 
of the illusory surface. These models have had a significant 
impact on the discussion of the nature of visual illusions be- 
cause they highlight the fact that illusory features, like the 
illusory surface of a Kanizsa figure, are observed as ex- 
tended image-like data structures, and therefore a com- 
plete model of the phenomenon must also produce a fully 
reified image-like spatial structure as its output. In fact, 
Grossberg’s concept of visual reification in his Boundary 
Contour System (Grossberg & Mingolla 1985) and Feature 
Contour System (Grossberg & Todorovi¢g 1988) were the 
original inspiration behind the perceptual modeling pro- 
posed in the present hypothesis. 

Although these models finally offer a reasonable account 
of perceptual experience (in two dimensions), they also 
demonstrate the profound limitations of a neural network 
architecture for perceptual representation because neural 
network theory is no different in principle than a template 
theory (Lehar 2003a), a concept whose limitations are well 
known. Grossberg and Mingolla (1985) account for col- 
linear illusory contour completion by way of specialized 
elongated receptive fields, tuned to detect and enhance 
collinearity. This concept works well enough for simple 
collinear boundary completion (as long as it remains re- 
stricted to two dimensions), but any attempt to extend this 
model to higher order perceptual processing runs headlong 
into a combinatorial explosion in required receptive fields 
(Lehar 2003a). For example, perceptual completion is ob- 
served not only for collinear alignments but it can also de- 
fine illusory vertices composed of two, three, or more edges 
that meet at a vertex (Lehar 2003a). Grossberg himself pro- 
posed an extension to his model equipped with “corner de- 
tector” receptive fields (Grossberg & Mingolla 1985), al- 
though this line of thought was subsequently quietly 
abandoned because, just as with the cells that perform 
collinear completion, the corner detectors would have to be 
provided at every location and every orientation across the 
visual field. To extend the model to account for T, V, Y, and 
X intersections, specialized receptive fields would have to 
be provided for each of those features at every location and 
at every orientation across the visual field. This combinato- 
rial explosion in the required number of specialized recep- 
tive fields does not bode well for neural network theory as 
a general principle of neurocomputation. 

The most serious limitation of Grossberg’s approach to 
perception is that, curiously, Grossberg and his colleagues 
did not extend their logic to the issue of three-dimensional 
spatial perception. In going from two dimensions to three, 
Grossberg no longer advocated explicit spatial filling-in, but 
instead represented the depth dimension by binocular dis- 
parity, using left and right eye image pairs (Grossberg 1987; 
1990; 1994). Although a stereo pair does encode depth in- 
formation, it does not do so in a volumetric manner because 
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it can only encode one depth or disparity value for every 
(x,y) point on the image. This makes it impossible for Gross- 
berg’s model to represent transparency with multiple depth 
values at a single (x,y) location, or to represent the experi- 
ence of empty space between the observer and a visible 
object. Moreover, it precludes the kind of volumetric fill- 
ing-in required to account, for example, for the three-di- 
mensional version of the Ehrenstein illusion constructed of 
a set of rods arranged radially around a circular void (Ware 
& Kennedy 1978). The filling-in processes in this illusion 
take place through the depth dimension, which produces 
an illusory percept of a glowing disk, hanging in space, as a 
volumetric spatial structure. If Grossberg’s argument for 
explicit filling-in of the two-dimensional illusions is at all 
valid, then that argument should apply equally to volumet- 
ric filling-in also. 

The reason Grossberg declined to extend his model into 
the third dimension is neurophysiologically motivated. For 
although Grossberg’s model is a de facto perceptual model, 
it is actually presented as a neural network model; that is, 
the computational units of the model represent actual neu- 
rons in the brain rather than perceptual entities. And this 
highlights the problem of perceptual modeling in neural 
network terms, for whenever there is a conflict between the 
perceptual phenomenon and our current understanding of 
neurophysiological principles, there is then a conflict be- 
tween the neural and the perceptual models of the phe- 
nomenon. In this case the percept is clearly volumetric, but 
the corresponding cortical neurophysiology is assumed to 
be two-dimensional. Another reason Grossberg was reluc- 
tant to extend his model into the third dimension is that, 
even for simple collinear completion, such an approach 
would require a volumetric block of neural elements each 
equipped with elongated receptive fields; and those fields 
must be replicated at every orientation in three dimensions 
and at every volumetric location across the entire volume of 
the perceptual representation — a notion that seems too im- 
plausible to contemplate, let alone the idea of T, V, Y, and X 
intersections defined in three dimensions. But until a map- 
ping has been established between the conscious experi- 
ence and the corresponding neurophysiological state, there 
is no way to verify whether the model has correctly repli- 
cated the psychophysical data. Because these models strad- 
dle the mind/brain barrier, they run headlong into the issue 
that Chalmers (1995) dubbed the “hard problem” of con- 
sciousness. Simply stated, even if we were to discover the 
exact neurophysiological correlates of conscious experi- 
ence, there would always remain a final explanatory gap be- 
tween the physiological and the phenomenal levels of de- 
scription. For example, if the activation of a particular cell 
in the brain were found to correlate with the experience of 
red at some point in the visual field, there would remain a 
vivid subjective quality, or quale, to the experience of red 
that is not in any way identical to any externally observable 
physical variable such as the electrical activity of a cell. In 
other words, there is a subjective experiential component 
of perception that can never be captured in a model ex- 
pressed in objective neurophysiological terms. 

Even more problematic for neural models of perception 
is the question of whether perceptual information is ex- 
pressed neurophysiologically in explicit or implicit form. 
For example, Dennett (1992) argued that the perceptual 
experience of a filled-in colored surface is encoded in more 
abstracted form in the brain, in the manner of an edge im- 
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age that records only the transitions along image edges. 
Support for this concept is seen in the retinal ganglion cells 
that respond only along spatial or temporal discontinuities 
in the retinal image and produce no response within regions 
of uniform color or brightness. This concept also appears to 
make sense from an information-theoretic standpoint, for 
uniform regions of color represent redundant information 
that can be compressed to a single value, as is the practice 
in image compression algorithms. These kinds of theoreti- 
cal difficulties have led many neuroscientists to simply ig- 
nore the conscious experience and to focus instead on the 
hard evidence of the neurophysiological properties of the 
brain. 


4. A perceptual modeling approach 


The quantification of conscious experience is not quite as 
hopeless as it might seem. Nagel (1974) suggested that we 
set aside temporarily the relation between mind and brain 
and devise a new method of objective phenomenology — in 
other words, quantify the structural features of the subjec- 
tive experience in objective terms without committing to 
any particular neurophysiological theory of perceptual rep- 
resentation. For example, if we quantify the experience of 
vision as a three-dimensional data structure, like a model of 
volumes and surfaces in a surrounding space to a certain 
perceptual resolution, this description could be meaningful 
even to a congenitally blind person or to an alien creature 
who had never personally experienced the phenomenon of 
human vision. Although this description could never cap- 
ture everything of that experience, such as the qualia of 
color experience, it would at least capture the structural 
characteristics of that subjective experience in an objective 
form that would be comprehensible to beings incapable of 
having those experiences. 

Chalmers (1995) extended this line of reasoning with the 
observation that the subjective experience and its corre- 
sponding neurophysiological state carry the same informa- 
tion content. On that ground, Chalmers proposed a princi- 
ple of structural coherence between the structure of 
phenomenal experience and the structure of objectively re- 
portable awareness, to reflect the central fact that con- 
sciousness and physiology do not float free of one another 
but cohere in an intimate way. In essence this is a restate- 
ment of the Gestalt principle of isomorphism, of which 
more in section 5. The connecting link between mind and 
brain therefore is information in information-theoretic 
terms (Shannon 1948) because the concept of information 
is defined at a sufficiently high level of abstraction to be in- 
dependent of any particular physical realization, and yet it 
is specified sufficiently to be measurable in any physical sys- 
tem given that the coding scheme is known. A similar ar- 
gument was made by Clark (1993, p. 50). Chalmers mod- 
erated his claim of the principle of structural coherence by 
stating that it is a hypothesis that is “extremely speculative.” 
However, the principle is actually solidly grounded episte- 
mologically because the alternative is untenable. If we ac- 
cept the fact that the physical states of the brain correlate 
directly with conscious experience, then the claim that con- 
scious experience contains more explicit information than 
does the physiological state on which it was based amounts 
to a kind of dualism that would necessarily involve some 
kind of nonphysical “mind stuff” to encode the excess in- 
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formation observed in experience that is not encoded by the 
physical state. Some theorists have even proposed a kind of 
hidden dimension of physical reality to house the unac- 
counted information in conscious experience (Harrison 
1989; Smythies 1994). 

The philosophical problems inherent in neural network 
models of perceptual experience can be avoided by propos- 
ing a perceptual modeling approach (Lehar 2003b), which 
models the conscious experience directly in the subjective 
variables of perceived color, shape, and motion, as opposed 
to neural modeling, where the conscious experience is mod- 
eled in the neurophysiological variables of neural activa- 
tions or spiking frequencies, or the like. The variables en- 
coded in the perceptual model therefore correspond to 
what philosophers call the sense-data or primitives of raw 
conscious experience, except that these variables are not 
supposed to be the sense-data themselves, they merely rep- 
resent the value or magnitude of the sense-data they are de- 
fined to represent. In essence this amounts to modeling the 
information content of subjective experience, which is the 
quantity that is common between mind and brain, thus al- 
lowing an objectively quantified description of a subjective 
experience. In fact, this approach is exactly the concept be- 
hind the description of phenomenal color space in the di- 
mensions of hue, intensity, and saturation, as seen in the 
CIE (Commission Internationale LEclairage) chromaticity 
space. The geometrical dimensions of that space have been 
tailored to match the properties of the subjective experi- 
ence of color as measured psychophysically, expressed in 
terms that are agnostic to any particular neurophysiological 
theory of color representation. 

Clark (1993) presented a systematic description of other 
sensory qualities in quantitative terms, based on this same 
concept of “objective phenomenology.” The thorny issue of 
the hard problem of consciousness is thus neatly side- 
stepped because the perceptual model remains safely on 
the subjective side of the mind/brain barrier, and therefore 
the variables expressed in the model refer explicitly to sub- 
jective qualia rather than to neurophysiological states of the 
brain. The problems of explicit versus implicit representa- 
tion are also neatly circumvented because those issues per- 
tain to the relation between mind and brain and so do not 
apply to a model that does not straddle the mind/brain bar- 
rier. For example, the subjective experience of a Necker 
cube is of a solid three-dimensional structure, and for that 
reason the perceptual model of that experience should also 
be an explicit three-dimensional structure. The sponta- 
neous reversals of the Necker cube, on the other hand, are 
experienced as a dynamic process, and on that ground 
should be represented in the perceptual model as a dy- 
namic process — that is, as a literal reversal of the solid 
three-dimensional structure. The issues of whether a per- 
ceived structure can be encoded neurophysiologically as a 
process or whether a perceived process can be encoded as 
a structure are therefore irrelevant to the perceptual 
model, which by definition models a perceived structure as 
a structure, and a perceived process as a process. 

This is of course only an interim solution, for eventually 
the neurophysiological basis of conscious experience must 
also be identified; nevertheless, the perceptual model does 
offer objective information about the informational content 
encoded in the physical mechanism of the brain. This is a 
necessary prerequisite to a search for the neurophysiologi- 
cal basis of conscious experience, for we must clearly cir- 


cumscribe that which we are to explain before we can at- 
tempt an explanation of it. This approach has served psy- 
chology well in the past, particularly in the field of color 
perception where the quantification of the dimensions of 
color experience led directly to great advances in our un- 
derstanding of the neurophysiology of color vision. The fail- 
ure to quantify the dimensions of spatial experience has 
been responsible for decades of futile debate about its neu- 
rophysiological correlates. I will show that application of 
this perceptual modeling approach to the realm of spatial 
vision opens a wide chasm between phenomenology and 
contemporary concepts of neurocomputation and thereby 
offers a valuable check on theories of perception based 
principally on neurophysiological concepts. 


5. The Gesialt principle of isomorphism 


The Gestalt principle of isomorphism represents a subtle 
but significant extension to Miiller’s psychophysical postu- 
late and to Chalmers’s principle of structural coherence. In 
the case of structured experience, equal dimensionality be- 
tween the subjective experience and its neurophysiological 
correlate implies similarity of structure or form. For exam- 
ple, the percept of a filled-in colored surface, whether real 
or illusory, encodes a separate and distinct experience of 
color at every distinct spatial location within that surface to 
a particular resolution. Each point of that surface is not ex- 
perienced in isolation but in its proper spatial relation to 
every other point in the perceived surface. In other words, 
the experience is extended in at least two dimensions, and 
therefore the neurophysiological correlate of that experi- 
ence must also encode at least two dimensions of percep- 
tual information. The mapping of phenomenal color space 
was established by the method of multidimensional scaling 
(Coren et al. 1994, p. 57) in which color values are ordered 
in psychophysical studies on the basis of their perceived 
similarity, to determine which colors are judged to be near- 
est to each other or which colors are judged to be between 
which other colors in phenomenal color space. A similar 
procedure could just as well be applied to spatial percep- 
tion to determine the mapping of phenomenal space. If two 
points in a perceived surface are judged psychophysically to 
be nearer to each other when they are actually nearer and 
farther when they are actually farther, and if other spatial 
relations such as betweenness are also preserved phenom- 
enally, then direct evidence is thereby provided that phe- 
nomenal space is mapped in a spatial representation that 
preserves those spatial relations in the stimulus. The out- 
come of this proposed experiment is so obvious it need 
hardly be performed. And yet its implication — that our phe- 
nomenal representation of space is spatially mapped — is 
not often considered in contemporary theories of spatial 
representation. 


5.1. Structural versus functional isomorphism 


The isomorphism required by Gestalt theory is not a strict 
structural isomorphism, a literal isomorphism in the phys- 
ical structure of the representation, but rather, it is merely 
a functional isomorphism, a behavior of the system as if it 
were physically isomorphic (Kéhler 1969, p. 92). This is be- 
cause the exact geometrical configuration of perceptual 
storage in the brain cannot be observed phenomenologi- 
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cally any more than the configuration of silicon chips on a 
memory card can be determined by software examination 
of the data stored within those chips. Nevertheless, the 
mapping between the stored perceptual image and the cor- 
responding spatial percept must be preserved, as in the case 
of the digital image, so that every stored color value is 
meaningfully related to its rightful place in the spatial per- 
cept. 

The distinction between structural and functional iso- 
morphism can be clarified with a specific example. Con- 
sider the spatial percept of a block resting on a surface, de- 
picted schematically in Figure 1A. The information content 
of this perceptual experience can be captured in a painted 
cardboard model built explicitly like Figure 1A, with ex- 
plicit volumes, bounded by colored surfaces, embedded in 
a spatial void. Because perceptual resolution is finite, the 
model should also be considered only to a finite resolution; 
that is, the infinite subdivision of the continuous space of 
the actual model world is not considered to be part of the 
model, which can only validly represent subdivision of 
space to the resolution limit of perception. The same per- 
ceptual information can also be captured in quantized or 
digital form in a volumetric or voxel (volume-pixel) image 
in which each voxel represents a finite volume of the cor- 
responding perceptual experience, as long as the resolution 
of this representation matches the spatial resolution of the 
percept itself; in other words, the size of the voxels should 
match the smallest perceivable feature in the correspond- 
ing spatial percept. Both the painted cardboard model and 
its quantized voxel equivalent are structurally or topo- 
graphically isomorphic with the corresponding percept; 
they have the same information content as the spatial per- 
cept that they represent. 

Consider now the flattened representation depicted in 
Figure 1B, which is identical to the model in Figure 1A ex- 
cept that the depth dimension is compressed relative to the 
other two dimensions, like a bas-relief. If the defined scale 
of the model (the length in the representation relative to 
the length that it represents) is also correspondingly com- 
pressed, as suggested by the compressed gridlines in the 
figure, then this model is also isomorphic with the percep- 
tual experience of Figure 1A. In other words the flattening 
of the depth dimension is not really registered in the model 
because the perceived cube spans the same number of grid- 
lines in Figure 1B (in all three dimensions) as it does in Fig- 
ure 1A, and therefore this flattened model encodes a non- 
flattened perceptual experience. Though this model is now 
no longer structurally isomorphic with the original percep- 
tual experience, it does remain topologically isomorphic, 
preserving neighborhood relations, as well as betweenness, 
and so forth. In a mathematical system with infinite resolu- 
tion, this model would encode the same information as the 
one in Figure 1A. However ina real physical representation 
there is always some limit to the resolution of the system, 
or how much information can be stored in each unit dis- 
tance in the model itself. In a representational system with 
finite resolution, therefore, the depth information in Fig- 
ure 1B would necessarily be encoded at a lower resolution 
than that in the other two dimensions. If our own percep- 
tual apparatus employed this kind of representation, this 
flattening would not be experienced directly; the only man- 
ifestation of the flattening of the representation would be a 
reduction in the resolution of perceived depth relative to 
the other two dimensions, making it more difficult to dis- 
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Figure 1. A. A volumetric spatial model, for example built of 


painted cardboard surfaces, is structurally isomorphic with a per- 
ceptual experience of a block resting on a surface if it has the same 
information content. B. If the model is compressed in one di- 
mension relative to the other two, the model can still be isomor- 
phic with the original percept if the representational scale of the 
model (indicated by the shaded gridlines) is also correspondingly 
compressed, although this is no longer a structural isomorphism 
but merely a topological isomorphism. C. The model can even be 
warped like the gyri and sulci of the cortical surface and remain 
isomorphic with the original percept. D. But a model composed 
of a small number of discrete depth planes is not isomorphic with 
the original percept because it no longer encodes the same infor- 
mation content. 


tinguish differences of perceived depth than differences of 
perceived height and width. 

Consider now the warped model depicted in Figure 1C, 
which is like the flattened model of Figure 1B with a wavy 
distortion applied, as if warped like the gyri and sulci of the 
cortical surface. This warped representation is also isomor- 
phic with the perceptual experience it represents for it en- 
codes the same information content as the flattened space 
in Figure 1B, although again this is a topological rather than 
a topographical isomorphism. The warping of this space 
would not be apparent to the percipient because the very 
definition of straightness is warped along with the space it- 
self, as suggested by the warped gridlines in the figure. In 
contrast, consider the flattened representation depicted in 
Figure 1D, where the perceptual representation has been 
segmented into discrete depth planes that distinguish only 
foreground from background objects. This model is no 
longer isomorphic with the perceptual experience it sup- 
posedly represents because, unlike this model, the percep- 
tual experience manifests a specific and distinct depth value 
for every point in each of the surfaces of the percept. Fur- 
thermore, the perceptual experience manifests an experi- 
ence of empty space surrounding the perceived objects, 
every point of which is experienced simultaneously and in 
parallel as a volumetric continuum of a certain spatial res- 
olution, whereas the model depicted in Figure 1D encodes 
only a small number of discrete depth planes. This kind of 
model therefore is inadequate as a perceptual model of the 
information content of conscious experience because the 
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dimensions of its representation are less than the dimen- 
sions of the experience it attempts to model. 

A functional isomorphism must also preserve the func- 
tional transformations observed in perception, and the ex- 
act requirements for a functional isomorphism depend on 
the functionality in question. For example, when a colored 
surface is perceived to translate coherently across per- 
ceived space, the corresponding color values in the per- 
ceptual representation of that surface must also translate 
coherently through the perceptual map. If that memory is 
discontinuous, like a digital image distributed across sepa- 
rate memory chips on a printed circuit board, then the per- 
ceptual representation of that moving surface must jump 
seamlessly across those discontinuities in order to account 
for the subjective experience of a continuous translation 
across the visual field. In other words, a functional isomor- 
phism requires a functional connectivity in the representa- 
tion, as if a structurally isomorphic memory were warped, 
distorted, or fragmented, but at the same time, the func- 
tional connectivity between its component parts were pre- 
served. Consider a representational mechanism, such as 
that shown in Figure 1A, equipped with additional compu- 
tational hardware capable of performing spatial transfor- 
mations on the volumetric image in the representation. The 
representational mechanism might be equipped with func- 
tions that could rotate, translate, and scale the spatial pat- 
tern in the representation on demand. This representation 
would thereby be invariant to rotation, translation, and 
scale, because the spatial pattern of the block itself would 
be encoded independent of its rotation, translation, and 
scale. The fact that an object in perception maintains its 
structural integrity and recognized identity despite rota- 
tion, translation, and scaling by perspective is clear evi- 
dence for this kind of invariance in human perception and 
recognition. If the warped model shown in Figure 1C were 
equipped with these same transformational functions, the 
warped representation would also be functionally isomor- 
phic with the non-warped representation as long as those 
transformations were performed correctly with respect to 
the warped geometry of that space. 

A functional isomorphism is even possible for a repre- 
sentation that is fragmented into separate pieces, if those 
pieces are wired together in such a way that they continue 
to perform the spatial transformations exactly as in the cor- 
responding undistorted mechanism. A functional isomor- 
phism can even survive in a volumetric representation 
whose individual elements or voxels are scrambled ran- 
domly across space, if the functional connections between 
those elements are preserved through the scrambling. The 
result is a representation that is neither topographically nor 
topologically isomorphic with the perceptual experience it 
represents. However, it does remains a volumetric repre- 
sentation, with an explicit encoding of each point in the rep- 
resented space to a particular spatial resolution, and it re- 
mains functionally isomorphic with the spatial experience 
that it represents, capable of performing coherent rotation, 
translation, and scaling transformations of the perceptual 
structures expressed in the representation. 

An explicit volumetric spatial representation capable of 
spatial transformation functions, as described above, is 
more efficiently implemented in either a topographically 
isomorphic form or a topologically isomorphic form, which 
require shorter and more orderly connections between ad- 
jacent elements in the representation. However, the argu- 


ment for structural or topological isomorphism is an argu- 
ment of representational efficiency and simplicity, rather 
than of logical necessity. On the other hand, a functional 
isomorphism is strictly required in order to account for the 
properties of the perceptual world as observed subjec- 
tively. The volumetric structure of visual consciousness 
and perceptual invariance to rotation, translation, and 
scale offer direct and concrete evidence for an explicit vol- 
umetric spatial representation in the brain, which is at least 
functionally isomorphic with the corresponding spatial ex- 
perience. 

A neurophysiological model of perceptual processing 
and representation should concern itself with the actual 
mechanism in the brain. In the case of a distorted repre- 
sentation (as in Fig. 1C), the warping of that perceptual 
map would be a significant feature of the model. A percep- 
tual model, on the other hand, is concerned with the struc- 
ture of the percept itself, independent of any warping of the 
representational manifold. Even for a representation that is 
functionally but not structurally isomorphic, a description 
of the functional transformations performed in that repre- 
sentation is most simply expressed in a structurally isomor- 
phic form, just as a panning or scrolling function in image 
data is most simply expressed as a spatial shifting of image 
data even when that shifting is actually performed in hard- 
ware ina non-isomorphic memory array. For that reason, 
the functional operation of a warped mechanism like Fig- 
ure 1C is most simply described as the operation of the 
functionally equivalent undistorted mechanism in Figure 
1A. In the present discussion, therefore, our concern will 
be chiefly with the functional architecture of perception, a 
description of the spatial transformations observed in per- 
ception, whatever form those transformations might take in 
the physical brain. And those transformations are most sim- 
ply described as if taking place in a physically isomorphic 
space. 

In the discussion that follows, the terminology “spatial 
representation,” “data expressed in spatial form,” “literal 
volumetric replica of the world inside your head,” “three- 
dimensional pattern of opaque-state units,” “explicit three- 
dimensional replica of the surface,” and “volumetric spatial 
medium,” will refer not to a topographically isomorphic 
model of space, as suggested in Figure 1A, but to a func- 
tionally isomorphic model of space like the warped model 
in Figure 1C, in which the explicit volumetric representa- 
tion is possibly warped and distorted but still encodes an ex- 
plicit value for every volumetric point in perceived space as 
well as the neighborhood relations between those values. 
This is in contrast to the more commonly assumed flattened 
or abstracted cortical representation depicted in Figure 
1D, where the volumetric mapping is no longer preserved. 


5.2. Second-order, complementary, and other 
paramorphisms 


The issue of isomorphism is so profoundly problematic for 
theories of perceptual representation that theorists have 
gone to no end of trouble in an effort to dispel the issue and 
to argue that isomorphism is not actually necessary. A care- 
ful examination of these proposals, however, reveals the 
naive realist assumptions on which they are founded. 
Shepard and Chipman (1970) argued that when we per- 
ceive a square, for example, there is no need for an internal 
perceptual replica of that square in the brain of the percip- 
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ient. They argued that we leam the appropriate use of 
words such as “square” from a verbal community that has 
access only to the public object and not to any such private 
image. If there is some internal event that corresponds to 
our experience of a square, whether it is the activation of a 
cell or cell assembly in the brain, our ability to form an as- 
sociation between this event and the word “square” re- 
quires only that this event have a regular relation to the ex- 
ternal object of causality, not of structural isomorphism. To 
insist, additionally, that these neurons must be spatially 
arranged in precisely the form of a square does not in the 
least help to explain how they come to trigger the naming 
response “square,” at least according to Shepard and Chip- 
man. 

As can be discerned from their brief introductory para- 
graph summarized above, Shepard and Chipman neatly 
turned the tables on the debate by characterizing the per- 
ception of a square as the issue of learning the naming re- 
sponse “square,” which is an issue of recognition rather 
than of perception. To be sure, recognition is an important 
aspect of perception, and the problem of learning a naming 
response is a formidable one that deserves further investi- 
gation. But the recognition response is by no means the 
same thing as the perceptual experience of the square as a 
continuous filled-in, square-shaped region of sense-data ex- 
perienced in the visual field. How can so intelligent and ed- 
ucated researchers come to make such a profound error in 
identification of the issue at hand? The answer is clear from 
their assertion that a verbal community has access only to 
the public object and not to any private image. This naive 
realist assumption is passed off casually as a statement of 
fact, but in fact it reveals an implicit commitment to the no- 
tion that the three-dimensional volumetric objects that we 
observe to occupy the space of our perceptual field are the 
actual objects themselves, and that therefore they need not 
be replicated or re-represented again in the brain. The fact 
that this assumption has gone unchallenged, and even 
largely unnoticed by the community at large, demonstrates 
how deeply the assumptions of naive realism have become 
entrenched in contemporary thought. 

Shepard (1981) made another attempt to dispel the issue 
of isomorphism by arguing for psychophysical complemen- 
tarity rather than isomorphism. Appropriately enough, 
Shepard cited that grand master of naive realism, B. F. 
Skinner, who argued that even if we were to discover a part 
of the brain in which the physical pattern of neural activity 
had the very same shape as the corresponding external ob- 
ject — say, a square — we would not in this way have made 
any progress toward explaining how the subject is able to 
recognize that object as a square, or to learn to associate to 
it a unique verbal response “square.” So again the issue of 
perception is confounded with the issue of recognition re- 
sponse. Skinner's statement is true enough, as far as it goes. 
But what Shepard and Skinner failed to acknowledge is that 
it would be very much harder to learn to recognize a square 
if you could not “see” it, that is, if you did not have direct 
access to an internal representation of the square as a 
square-shaped sense-datum to associate with the appropri- 
ate recognition response. To claim that we can experience 
the square without such an internal replica is just plain 
magic. Furthermore, until we do discover a part of the brain 
in which the physical pattern of neural activity (or some 
other physically measurable quantity) has the very same 
shape as the corresponding external object, the phenome- 
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nal aspect of that volumetric spatial structure remains as a 
nomological dangler, something that is experienced as a 
spatial picture, something that is clearly distinct from the 
actual square in the real world (especially when that square 
is illusory), but something that does not actually exist in any 
space known to science. Like the Behaviorists before him, 
Shepard attempted to discount the entire edifice of con- 
scious experience as if it simply did not exist as a scientific 
entity. 

There is a further difficulty with the notion of psy- 
chophysical complementarity. Shepard (1981) argued that 
the relation of the mental representation to the external ob- 
ject it represents might be one of complementarity, rather 
than one of similarity or resemblance. Just as a lock has a 
hidden structure that is to some extent complementary to 
the visible contour of the key that fits it, the internal struc- 
ture uniquely activated by a given object must have a struc- 
ture that somehow meshes with the pattern manifested by 
its object; in other words, the “shape” of the representation 
is complementary to, rather than isomorphic with, the ob- 
ject that it represents. But, again, this notion of perceptual 
representation is only coherent from a naive realist per- 
spective. If we interpret this argument from an indirect per- 
ceptual view, it would have to be that the square shape we 
experience in immediate consciousness is complementary 
to the external square, which is beyond our direct experi- 
ence. In other words, the real “square” in the external world 
is not actually square as we observe it to be, but rather it 
would have to be somehow complementary to the square 
shape we observe in conscious experience, an idea that is 
obviously absurd. 

In yet another, somewhat different, defense of naive re- 
alism, Shepard (1981, p. 292) argued that the relation be- 
tween the external object and its internal representation 
might be a kind of paramorphism rather than isomorphism, 
as seen for example in the Fourier transform of an image, 
which encodes all of the information in a spatial image but 
in a very abstract nonspatial form. Again, this argument 
is founded on the naive assumption that the world we 
see around us is the world itself, and that therefore the 
paramorphic representation of that world is not identified 
as the image of the world we see around us but as our ver- 
bal or conceptual recognition of that world. If the percep- 
tual brain did indeed employ a Fourier representation in- 
stead of a spatial one, then the world we see around us 
would necessarily appear in the form ofa Fourier transform 
rather than as a spatial structure, which, again, is obviously 
absurd. The fact that the world around us appears as a vol- 
umetric spatial structure is direct and concrete evidence for 
a spatial representation in the brain. What is most interest- 
ing about this issue is that Shepard clearly did not fully com- 
prehend the position that he challenged, and therefore his 
criticisms of isomorphism inevitably missed the mark. 

Steven Palmer (1999) on the other hand struck at the 
very heart of the issue of isomorphism. Palmer drew a dis- 
tinction between two different aspects of conscious experi- 
ence, the intrinsic qualities of experiences themselves ver- 
sus the relational structure that holds among those 
experiences. The intrinsic qualities, such as the color qualia 
in the experience of color, are in principle impossible to 
communicate from one mind to another, and therefore they 
are inaccessible to science (except through phenomenol- 
ogy), a restriction that Palmer calls the subjectivity barrier. 
All that can be communicated about conscious experience 
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is the relational structure that holds among those experi- 
ences. In the case of color experience, for example, subjects 
say that orange is more similar to red than it is to green or 
blue, and that aqua is experienced as intermediate between 
green and blue, and so forth. It was exactly these relational 
facts of color experience that were used to define the color 
solid in the CIE chromaticity diagram. A relational struc- 
ture like the color solid encodes a great deal of information 
implicitly about the relations between its variables in a man- 
ner that is practically impossible to express as explicit rela- 
tions because the number of binary, trinary, and other rela- 
tions between colors implicitly expressed in the color solid 
is so astronomical as to defy any kind of exhaustive listing 
or discrete associative links. And yet all of those relations 
are evidently available to the psychophysical subject when 
making phenomenal color judgments. This strongly sug- 
gests that the variables of phenomenal color experience are 
encoded in the brain as a relational structure whose infor- 
mation content is identical to that of the color solid, rather 
than as a list of the astronomical number of relations be- 
tween individual colors that are expressed implicitly within 
the color solid. 

The subjectivity barrier is often cited as an insurmount- 
able obstacle to meaningful phenomenological examination 
of brain states. But Palmer observed that the isomorphism 
constraint goes both ways. Not only is it impossible to ex- 
press intrinsic color experience in objective external terms, 
but even if there were some way to quantify the intrinsic 
qualities of experience, it would then be impossible to infer 
the structure of the brain from that intrinsic information. 
The relational structure, on the other hand, does offer di- 
rect evidence for the dimensions of color experience as ex- 
pressed in the physical brain because relational information 
is the only information that can cross the subjectivity bar- 
rier in either direction. Palmer’s analysis of isomorphism 
has profound implications not only for color perception but 
also for the perception of space — although curiously 
Palmer avoided discussing the issue of spatial perception, 
presumably because including such a controversial thesis 
might imperil the chance of having his paper published. But 
a spatial percept, like that of a square, is clearly a relational 
structure in the sense that every point of the percept is pre- 
sented simultaneously in proper spatial relation to every 
other point in the square. In other words, our experience of 
a square is of a spatial structure, and therefore the infor- 
mation encoded in spatial perception is an explicit spatial 
one, whether expressed in topographical or only topologi- 
cal isomorphic form. 


6. The dimensions of conscious experience 


The phenomenal world is composed of solid volumes, 
bounded by colored surfaces, embedded in a spatial void. 
Every point on every visible surface is perceived at an ex- 
plicit spatial location in three dimensions (Clark 1993; 
Lehar 2003b), and all of the visible points on a perceived 
object, such as a cube ora sphere or this page, are perceived 
simultaneously in the form of continuous surfaces in depth. 
The perception of multiple transparent surfaces, as well as 
the experience of empty space between the observer anda 
visible surface, reveals that multiple depth values can be 
perceived at any spatial location. I propose to model the in- 
formation in perception as a computational transformation 


from a two-dimensional colored image (or two images in 
the binocular case) to a three-dimensional volumetric data 
structure in which every point can encode either the expe- 
rience of transparency or the experience of a perceived 
color at that location. The appearance of a color value at 
some point in this representational manifold corresponds 
by definition to the subjective experience of that color at the 
corresponding point in phenomenal space. If we can de- 
scribe the generation of this volumetric data structure from 
the two-dimensional retinal image as a computational 
transformation, we will have quantified the information 
processing that is apparent in perception as a necessary pre- 
requisite to the search for a neurophysiological mechanism 
that can perform that same transformation. 


6.1. The Cartesian theatre and the homunculus problem 


This “picture-in-the-head” or “Cartesian theatre” concept 
of visual representation has been criticized on the grounds 
that there would have to be a miniature observer to view 
this miniature internal scene, resulting in an infinite regress 
of observers within observers (Dennett 1991; 1992; O’Re- 
gan 1992; Pessoa et al. 1998). In fact, there is no need for 
an internal observer of the scene because the internal rep- 
resentation is simply a data structure like any other data in 
a computer, except that these data are expressed in spatial 
form (Earle 1998; Lehar 2003b; Singh & Hoffman 1998). If 
a picture in the head required a homunculus to view it, then 
the same argument would hold for any other form of infor- 
mation in the brain, which would also require a homuncu- 
lus to read or interpret that information. But, in fact, any in- 
formation encoded in the brain needs only to be available 
to other internal processes rather than to a miniature copy 
of the whole brain. The fact that the brain does go to the 
trouble of constructing a full spatial analog of the external 
environment merely suggests that it has ways to make use 
of these spatial data. For example, field theories of naviga- 
tion have been proposed (Gibson & Crooks 1938; Koffka 
1935, pp. 42-46) in which perceived objects in the per- 
ceived environment exert spatial fieldlike forces of attrac- 
tion and repulsion, drawing the body toward attractive per- 
cepts and repelling it from aversive percepts, as a spatial 
computation taking place in a spatial medium. 

If the idea of an explicit spatial representation in the 
brain seems to “fly in the face of what we know about the 
neural substrates of space perception” (Pessoa et al. 1998, 
Authors’ Response sect. R3.2, p. 789), it is our theories of 
spatial representation that are in urgent need of revision, 
for to deny the spatial nature of the perceptual representa- 
tion in the brain is to deny the spatial nature so clearly evi- 
dent in the world we perceive around us. To paraphrase 
Descartes, it is not only the existence of myself that is veri- 
fied by the fact that I think, but when I experience the vivid 
spatial presence of objects in the phenomenal world, those 
objects are certain to exist, at least in the form of a subjec- 
tive experience, with the properties I experience them to 
have: location, spatial extension, color, and shape. I think 
them, therefore they exist (Price 1932, p. 3). All that re- 
mains uncertain is whether those percepts exist also as ob- 
jective external objects as well as internal perceptual ones, 
and whether their perceived properties correspond to ob- 
jective properties. But their existence and fully spatial na- 
ture in my internal perceptual world are beyond question if 
I experience them so, even if only as a hallucination. 
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6.2. Bounded nature of the perceptual world 


The idea of perception as a literal volumetric replica of the 
world inside your head immediately raises the question of 
boundedness: How can an explicit spatial representation 
encode the infinity of external space in a finite volumetric 
system? The solution to this problem can be found by in- 
spection, for phenomenological examination reveals that 
perceived space is not infinite but is bounded (Lehar 
2003b). This can be seen most clearly in the night sky, where 
the distant stars produce a domelike percept that presents 
the stars at equal distance from the observer, and that dis- 
tance is perceived to be less than infinite. The lower half of 
perceptual space is usually filled with a percept of the 
ground underfoot, but it too becomes hemispherical when 
viewed from far enough above the surface, as from an air- 
plane or a hot air balloon. Thus the dome of the sky above 
and the bow! of the earth below define a finite approxi- 
mately spherical space (Heelan 1983) that encodes dis- 
tances out to infinity within a representational structure 
that is both finite and bounded. Although the properties of 
perceived space are approximately Euclidean near the 
body, there are peculiar global distortions evident in per- 
ceived space that provide clear evidence of the phenome- 
nal world being an internal rather than an external entity. 


6.3. The phenomenon of perspective 


Consider the phenomenon of perspective, as seen for ex- 
ample when standing on a long straight road that stretches 
to the horizon in a straight line in opposite directions. The 
sides of the road appear to converge to a point both up 
ahead and back behind, but, while converging, they are also 
perceived to pass to either side of the percipient, and, at the 
same time, the road is perceived to be straight and parallel 
throughout its entire length. This property of perceived 
space is so familiar in everyday experience as to seem totally 
unremarkable. And yet this most prominent violation of 
Euclidean geometry offers clear evidence for the non-Eu- 
clidean nature of perceived space, for the two sides of the 
road must in some sense be perceived as being bowed, and 
yet they are also perceived as being straight. This can only 
mean that the space within which we perceive the road to 
be embedded must itself be curved. In fact, the observed 
warping of perceived space is exactly the property that al- 
lows the finite representational space to encode an infinite 
external space. This property is achieved by using a variable 
representational scale, that is, the ratio of the physical dis- 
tance in the perceptual representation relative to the dis- 
tance in external space that it represents. This scale is ob- 
served to vary as a function of distance from the center of 
our perceived world, such a way that objects close to the 
body are encoded at a larger representational scale than ob- 
jects in the distance, and beyond a certain limiting distance 
the representational scale, at least in the depth dimension, 
falls to zero — that is, objects beyond a certain distance lose 
all perceptual depth. This is seen, for example, when the 
sun and moon and distant mountains appear as if cut out of 
paper and pasted against the dome of the sky. 

The distortion of perceived space is suggested in Figure 
2, which depicts the perceptual representation for a man 
walking down a road. The phenomenon of perspective is by 
definition a transformation defined from a three-dimen- 
sional world through a focal point to a two-dimensional sur- 
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Figure 2. The perceptual representation of a man walking down 
along straight road. The sides of the road are perceived to be par- 
allel and equidistant throughout their length, and at the same time 
they are perceived to converge to a point both up ahead and be- 
hind, and that point is perceived at a distance that is less than in- 
finite. This peculiar violation of Euclidean geometry is perhaps 
the best evidence for the internal nature of the perceived world, 
for it shows evidence, out in the world around us, of the perspec- 
tive projection due to the optics of the eye. 


face. The appearance of perspective on the retinal surface 
therefore is no mystery and is similar in principle to the im- 
age formed by the lens in a camera. What is remarkable in 
perception is that perspective is not observed on a two-di- 
mensional surface but is somehow embedded in the three- 
dimensional space of our perceptual world. Nowhere in the 
objective world of external reality is there anything that is 
remotely similar to the phenomenon of perspective as we 
experience it phenomenologically, where a perspective 
foreshortening is observed not on a two-dimensional image 
but in three dimensions on a solid volumetric object. The 
appearance of perspective in the three-dimensional world 
we perceive around us is perhaps the strongest evidence for 
the internal nature of the world of experience, for it shows 
that the world that appears to be the source of the light that 
enters our eye must actually be downstream of the retina, 
for it exhibits the traces of perspective distortion imposed 
by the lens of the eye, although in a completely different 
form. 

This view of perspective offers an explanation for another 
otherwise paradoxical but familiar property of perceived 
space whereby more distant objects are perceived to be 
both smaller and, at the same time, undiminished in size. 
This corresponds to the difference in subjects’ reports de- 
pending on whether they are given objective instruction or 
projective instruction (Coren et al. 1994, p. 500) in how to 
report their observations, for both types of information are 
available perceptually. This duality in size perception is of- 
ten described as a cognitive compensation for the fore- 
shortening of perspective, as if the perceptual representa- 
tion of more distant objects is indeed smaller but is 
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somehow labeled with the correct size as some kind of sym- 
bolic tag representing objective size attached to each object 
in perception. However, this kind of explanation is mis- 
leading, for the objective measure of size is not a discrete 
quantity attached to individual objects but is more of a con- 
tinuum, or gradient of difference between objective and 
projective size, that varies monotonically as a function of 
distance from the percipient. In other words, this phenom- 
enon is best described as a warping of the space itself within 
which the objects are represented, so that objects that are 
warped coherently along with the space in which they are 
embedded appear undistorted perceptually. The mathe- 
matical form of this warping will be discussed in more de- 
tail in section 8.7 below. 


6.4. The embodied percipient 


This model of spatial representation emphasizes another 
aspect of perception that is often ignored in models of vi- 
sion: Our percept of the world includes a percept of our 
own body within that world, and our body is located at a 
very special location at the center of that world, and it re- 
mains at the center of perceived space even as we move 
about in the external world. Perception is embodied by its 
very nature, for the percept of our body is the only thing 
that gives an objective measure of scale in the world, anda 
view of the world around us is useless if it is not explicitly 
related to our body in that world. The little man at the cen- 
ter of the spherical world of perception therefore is not a 
miniature observer of the internal scene but is itself a spa- 
tial percept, constructed of the same perceptual material as 
the rest of the spatial scene, for that scene would be in- 
complete without a replica of the percipient’s own body in 
his perceived world. Gibson (1979) was right, therefore, in 
his emphasis on the interaction of the active organism with 
its environment. Gibson’s only error was the epistemologi- 
cal one of failing to recognize that the organism and its en- 
vironment that are active in perception, are themselves in- 
ternal perceptual replicas of their external counterparts. It 
was this epistemological confusion that led to the bizarre as- 
pects of Gibson’s otherwise valuable theoretical contribu- 
tions. 


6.5. The ultimate question of consciousness 


Indirect realism offers direct evidence for a spatial repre- 
sentation in the brain, but there remains one final question 
regarding the ultimate nature of consciousness. Even if 
there is a spatial representation in the brain, why should it 
be conscious of itself? Why should it not behave much like 
a machine that performs its function using either a spatial 
or a symbolic principle of computation but, presumably, 
performs its function without any conscious experience of 
what it is doing? Why should human consciousness be any 
different? 

But there is a large unstated assumption implied in the 
very framing of this consciousness question. The assump- 
tion is that a machine could not possibly be conscious. This 
assumption is generally taken for granted because the al- 
ternative, that everything in the universe must have some 
primitive level of consciousness, seems so absurd from the 
outset that, like solipsism, we tend to discount it even if we 
cannot disprove it on logical grounds. But can we really be 
sure that this alternative is so absurd? Obviously, like solip- 


sism, the possibility of panpsychism or, more likely, panex- 
perientialism (Chalmers 1995; Rosenberg 2003) is a ques- 
tion that might never be provable one way or the other. 
Nevertheless, it is of vital importance that we get this ques- 
tion right, because if we come down on the wrong side of 
this paradigmatic fence, that will necessarily throw all the 
rest of our philosophy completely out of kilter. 

If we accept the materialist view that mind is a physical 
process taking place in the physical mechanism of the brain, 
and since we know that mind is conscious, then we already 
have direct and incontrovertible evidence that a physical 
process taking place in a physical mechanism can under 
certain conditions be conscious. Now, it is true that the 
brain is a very special kind of mechanism. But what makes 
the brain so special is not its substance, for it is made of the 
ordinary substance of matter and energy. What sets the 
brain apart from normal matter is its complex organization. 
The most likely explanation, therefore, is that what makes 
our consciousness special is not its substance but its com- 
plex organization. The fundamental “stuff” of which our 
consciousness is composed — the basic qualia of color and 
spatial extension — are apparently common with the qualia 
of children, as far back as I can remember; although I also 
remember a less complex organization of my experiences as 
achild. It is also likely, on logical grounds, that animals have 
some kind of conscious qualia because the information en- 
coded in their perceptual state cannot be experienced 
without some kind of quale, or carrier to express that infor- 
mation in the form of experience. If the experience of mind 
is identified as the functioning of the physical brain, then 
the functioning animal brain must also involve an experi- 
ence of mind. Whether the subjective qualia of different 
species, or even of different individuals of our own species, 
are necessarily the same as ours experientially is a question 
that is difficult, even impossible in principle, to answer de- 
finitively. But the simplest, most parsimonious explanation 
is that our own conscious qualia evolved from those of our 
animal ancestors, and differ from those earlier forms more 
in their level of complex organization than in their funda- 
mental nature. 

The natural reluctance we all feel to extending con- 
sciousness to our animal ancestors, and even more so to 
plants or to inanimate matter, is a stubborn legacy of our an- 
thropocentric past. But the history of scientific discovery 
has been characterized by a regular progression of an- 
throdecentralization, demoting humans from the central 
position in the universe under the personal supervision of 
God, to lost creatures on the surface of a tiny blip of matter 
orbiting a very unremarkable star among countless billions 
of stars in an unremarkable galaxy amongst countless bil- 
lions of other galaxies as far as the telescopic eye can see. 
Modern biology has now discovered that there is no vital 
force in living things, but only a complex organization of the 
ordinary matter of the universe, following the ordinary laws 
of that universe. There is no reason on earth why con- 
sciousness should not also be considered to be a manifes- 
tation of the ordinary matter of the universe following the 
ordinary laws of that universe, although expressed in a com- 
plex organization in the case of the human brain. A claim to 
the contrary would necessarily fall under the category of an 
extraordinary claim, which, as Carl Sagan pointed out, 
would require extraordinary evidence for it to be accepted 
by reasonable men. 

When we examine the chain of biocomplexity from the 
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simplest pure chemical to the most complex human brain, 
there is a continuous progression from single atoms, to 
simple compound molecules, to complex organic mole- 
cules, to proteins and DNA, to viruses, to simple single- 
celled organisms, and all the way up the evolutionary chain 
to the brain of man. If we are to claim that consciousness is 
uniquely human, or unique to animals above a certain com- 
plexity, then there would necessarily be some kind of abrupt 
transition along that progression where that consciousness 
comes suddenly into existence, and that abrupt transition 
would occur both for the individual during gestation and for 
the species during evolution. The claim that consciousness 
is unique to humans, or to animals, or to living creatures, is 
bedeviled by the fact that there are always transitionary 
forms to be found that are intermediate between humans 
and animals, between animals and plants, and between liv- 
ing and nonliving creatures such as hypercomplex mole- 
cules like viruses; and there is also a continuous progression 
during gestation from fertilized egg to full human body. If 
we posit that consciousness appears abruptly at any one of 
these transitions where the only observed difference is a 
slight increase in complexity of organization, then we again 
lapse into nomological danglers and vital force because 
the postulated conscious quality that supposedly appears 
abruptly at that point is undetectable to science, and there- 
fore it is a quality of the supervenient spirit world rather 
than anything knowable by, or demonstrable to, science. 
Surely the time has come to finally accept the full implica- 
tions of Darwin’s theory of evolution and acknowledge the 
fact that our nature and our consciousness are not of a sep- 
arate spiritual realm but are composed of the very same ma- 
terial substance and energy of which the rest of the universe 
is composed. 

The inescapable conclusion is that all matter and energy 
have some kind of primal protoconsciousness, what 
Chalmers (1995) calls “panexperientialism’” to distinguish it 
from panpsychism, the view that everything is conscious in 
any human kind of sense. The more plausible panexperien- 
tialism posits merely that there exists a very simple proto- 
consciousness in inanimate matter that is a fundamental 
property of that matter. For inanimate matter, this proto- 
consciousness is something so simple and primitive that we 
would hardly recognize it as consciousness at all. And yet 
when this protoconsciousness is organized in the right 
manner in a human brain, it gives rise to the wonderful 
splendor of human consciousness. We are not external ob- 
servers of the physical universe, rather we ourselves are 
part of that universe and our experience is a tiny fragment 
of the experience of the larger universe around us, although 
expressed in a very much more complex form in the human 
brain. This way of describing consciousness is the only true 
monism that really equates mind with the functioning of 
physical matter, without recourse to nomological danglers 
and spiritual mumbo jumbo. 

This identity relation between mind and matter casts a 
new light on Searle’s (1997) assertion that “a computer is 
not even a computer to a computer.” What would the con- 
sciousness of a computer be like, if a computer did have 
consciousness? Consider the hypothesis that consciousness 
is a manifestation of forces and energy, or energetic wrin- 
kles in space-time, or what Rosenberg (2003) called mani- 
festations of causality in the physical world. The conscious- 
ness of a computer would thereby correspond to the 
patterns of energy in its chips and wires. In a digital com- 


390 BEHAVIORAL AND BRAIN SCIENCES (2008) 26:4 


puter that consciousness would be a very binary affair, and 
itis also in the very nature of digital computation that com- 
plex calculations are divided into a number of very simple 
steps, each of which can be computed independent of the 
problem as a whole. The consciousness of a computer 
would thereby be a very fragmented kind of thing, with 
each flip-flop or logic gate experiencing only the energy 
state in its local inputs and outputs because those are the 
only forces that influence the local logic gate. There is a very 
different kind of energy structure in an analog spatial sys- 
tem like a soap bubble, whose entire surface is under ten- 
sion against the outward pressure of the captured air. A 
push on any point of the bubble has an immediate influence 
on the bubble as a whole, on the entire gestalt, whose causal 
structure works in an emergent manner to try to restore the 
spherical shape. If a soap bubble has any form of primal 
consciousness, that protoconsciousness would be of an elas- 
tic spherical form under stress, as a unitary gestalt. 

It is curious that in his Chinese room analogy, Searle 
(1980) assumed a fragmented, rule-based mechanism as his 
model of conscious experience, because in his analogy the 
Chinese translation is performed step-by-step, very much 
like the computation in a digital computer. No wonder 
there is no emergent global consciousness from such a 
fragmented computational analogy. But does a globally 
integrated analog structure like a soap bubble have a cor- 
responding global consciousness independent of the indi- 
vidual consciousnesses of its constituent parts? And does 
that larger consciousness include the consciousnesses of its 
individual parts? The answers to these questions can be 
found by inspection of our own consciousness. 

By the fact that we ourselves have global consciousness, 
we can infer that larger global phenomena in the brain do 
give rise to global emergent consciousness that takes the 
form we observe in the perceived world around us. And that 
global consciousness does not appear to include a con- 
sciousness of its individual elements, for we are completely 
unaware of the component electrons, molecules, and neu- 
rons of our own physical brain that must be responsible for 
that global percept. Our personal conscious experience is 
therefore confined to an awareness of the spatial structures 
of the patterns of energy in our brain, although presumably 
there would also be many more independent and discon- 
nected consciousnesses in the energy structures of our 
physical body of which we are not directly aware, and most 
likely there are also multiple independent conscious enti- 
ties within our own brain, which make up the “unconscious 
mind,” and of which our central narrative consciousness is 
not directly aware. 

Consider the experience of swallowing food. I am con- 
scious of the inside of my mouth as a vivid three-dimen- 
sional structure “colored” by sensations of taste and texture, 
warmth and cold. But this spatial consciousness terminates 
abruptly at the threshold of my throat, beyond which my 
spatial consciousness of the food is abruptly cut off. The rest 
of my alimentary canal performs wavelike motions of peri- 
staltic contraction, very much like the kind of manipulation 
that occurs consciously in my mouth, but these motions are 
all beyond my own personal conscious awareness. Is my al- 
imentary canal conscious of itself, or does it perform its 
function totally in the absence of conscious experience? It 
seems that conscious experience has a direct functional 
role, because my consciousness of my own mouth helps me 
to chew the food and direct it intelligently down my throat 
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without choking. If the food I swallowed was a hot and spicy 
vindaloo curry, I know in an indirect way that my stomach is 
feeling the burning pain because I can feel it churning and 
grinding in protest, although I cannot feel its pain directly, 
only remotely, like a loud argument heard through the wall 
in an adjacent motel room. And the next morning, as the vin- 
daloo curry passes another abrupt threshold portal, I be- 
come suddenly aware of the pain again as part of my own 
personal experience. The simplest explanation therefore is 
that my alimentary canal has a similar conscious experience; 
it feels the waves of peristaltic contraction, which are its own 
conscious wavelike thoughts, just as I feel the inside of my 
mouth, although, unlike my central narrative consciousness, 
presumably the alimentary consciousness is not burdened 
by memories or aspirations or any real self-consciousness ex- 
cept of itself as a spatial structure and of the vital imperative 
to propel arriving food farther down the pipeline. This hy- 
pothesis is supported by the fact that there exist rare indi- 
viduals who have conscious control over their own bowel 
functions, such that they can consciously control their own 
alimentary peristaltic contractions just as we can control the 
contractions of our mouths. And that extra level of control is 
accompanied by a conscious experience of their alimentary 
canal in places where we have none. In fact, it is not clear if 
it is even intelligible to have control of a body part without 
having some kind of consciousness of that control. 

There is not, therefore, a single “bridge locus” that is the 
only place in the brain where consciousness occurs, but 
rather there is one global representational mechanism 
which has verbal and cognitive access to the components of 
ordinary consciousness including memories and aspira- 
tions, and then there are countless additional independent 
conscious energy structures disconnected from our global 
or narrative consciousness of which we remain personally 
unaware. Each of those islands of consciousness has an iso- 
lated experience of its own energy structure. 

If consciousness is indeed identical to energy structure, 
then the spherical bubble can be conscious of its own spher- 
ical form, although it has neither memory nor aspirations, 
nor any kind of understanding except an understanding of 
its own spherical energy structure. How then does human 
consciousness come to be aware not only of its own struc- 
ture but also that of the external environment around it be- 
yond the bounds of the physical brain? It does so by con- 
structing a much more complex and elaborate bubble 
structure in the human brain, composed of patterns of elec- 
trochemical energy that take the form of a replica of the ex- 
ternal world, complete with a replica of our own body at the 
center of that representational space. So, in answer to 
Searle’s contention, the computer too could acquire a con- 
sciousness of itself, if it were loaded with a representation 
of itself. The pattern of that representation in the computer 
would thereby appear as a computer to the computer. Of 
course, like us, the computer would not notice that what it 
was seeing was not really an image of its real self, as viewed 
from the outside, but merely a miniature representation of 
itself that would be entirely contained within itself, because 
its computational consciousness could not extend beyond 
the confines of its computational brain. The computer 
would not know that everything of which it was aware was 
actually surrounded by the larger physical computer, which 
in turn was composed of entirely different and independent 
sets of conscious energy structures in the physical struc- 
tures of its frame and screws and power supply. 


If this notion of panexperientialism, or protoconscious- 
ness of inanimate matter, sounds bizarre and far-fetched, 
we should bear in mind that whatever the ultimate solution 
to the mind-brain quandary, it is sure to do considerable vi- 
olence to our normal, everyday, commonsense notions of 
reality. When it comes to these fundamental issues of exis- 
tence, our intuitive instincts are almost certain to fail us, 
and therefore every alternative should be given serious con- 
sideration, however implausible it might at first seem intu- 
itively. For, as intuitively incredible as the notion of panex- 
perientialism might seem, the alternatives are all fraught 
with even more profound philosophical paradoxes and con- 
tradictions. But whatever our theoretical inclinations on the 
ultimate question of consciousness, it is important to point 
out that this is a separate and independent issue from the 
question of whether the internal representation of the brain 
is spatial or symbolic. Whichever way the answer to the ul- 
timate question goes, whether consciousness is uniquely 
human or is shared with the living and nonliving worlds, un- 
less we wish to believe in some magical nomological dan- 
gler that extends mind halfway into the spirit world, we 
must face the observational fact that there is a spatial rep- 
resentation in the brain. 


7. The Gesialt properties of perception 


One of the most formidable obstacles facing computational 
models of the perceptual process is that perception exhibits 
certain global Gestalt properties such as emergence, reifi- 
cation, multistability, and invariance that are difficult to ac- 
count for either neurophysiologically or even in computa- 
tional terms such as computer algorithms. The ubiquity of 
these properties in all aspects of perception, as well as their 
preattentive nature, suggests that Gestalt phenomena are 
fundamental to the nature of the perceptual mechanism. I 
propose that no useful progress can possibly be made in our 
understanding of neural processing until the computational 


principles behind Gestalt theory have been identified. 


7.1. Emergence 


Figure 3 shows a picture that is familiar in vision circles, for 
it reveals the principle of emergence in a most compelling 
form. The picture appears initially as a random pattern of 
irregular shapes, but a remarkable transformation is ob- 
served in this percept as soon as one recognizes the subject 
of the picture as a dalmation in the patchy sunlight under 
overhanging trees. What is remarkable about this percept 
is that the dog is perceived so vividly despite the fact that 
much of its perimeter is missing. Furthermore, visual 
edges, which form a part of the perimeter of the dog, are 
locally indistinguishable from other less significant edges. 
Therefore, any local portion of this image does not contain 
the information necessary to distinguish significant from in- 
significant edges. 

Although Gestalt theory did not offer any specific com- 
putational mechanism to explain emergence in visual per- 
ception, Koffka (1935) suggested a physical analogy of the 
soap bubble to demonstrate the operational principle be- 
hind emergence. The spherical shape of a soap bubble is 
not encoded in the form of a spherical template or abstract 
mathematical code, but rather, that form emerges from the 
parallel action of innumerable local forces of surface ten- 
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Figure 3. The dog picture is familiar in vision circles for it 
demonstrates the principle of emergence in perception. The local 
regions of this image do not contain sufficient information to dis- 
tinguish significant form contours from insignificant noisy edges. 
As soon as the picture is recognized as that of a dog in the dappled 
sunshine under trees, the contours of the dog pop out perceptu- 
ally, filling in visual edges in regions where no edges are present 
in the input. 


sion acting in unison. The characteristic feature of emer- 
gence is that the final global form is not computed in a sin- 
gle pass but continuously, like a relaxation to equilibrium in 
a dynamic system model. In other words, the forces acting 
on the system induce a change in the system configuration, 
and that change in turn modifies the forces acting on the 
system. The system configuration and the forces that drive 
it are changing continuously in time until equilibrium is at- 
tained, at which point the system remains in a state of dy- 
namic equilibrium. Even at this point, its static state belies 
a dynamic balance of forces ready to spring back into mo- 
tion as soon as the balance is upset. 

Emergence is actually the issue that inspired Davidson’s 
(1970) theory of anomalous monism. Davidson argued 
(p. 247) that mental events resist capture in the nomologi- 
cal net of physical theory, for mentalistic propositions do 
not display the law-like character of physical ones. David- 
son asserted (p. 248) that “there are no strict deterministic 
laws on the basis of which mental events can be predicted 
and explained,” and this is the principle of the anomalism 
of the mental. But Wolfgang Kéhler (1924) showed that in 
fact there is no magic in emergence; rather, emergence is a 
common property of certain kinds of physical systems, such 
as the soap bubble taking on its spherical shape, or water 
seeking its own level in a vessel, or global weather patterns 
defying lawful prediction based on their present state. To 
insist that mind supervenes on the brain in some mysteri- 
ous way is like saying that the soap bubble supervenes on 
soapy water, or that the water level supervenes on the body 
of water in a vessel, or that global weather patterns super- 
vene on the earth’s physical atmosphere. But this is no dif- 
ferent than saying that these are emergent processes that 
are already the simplest model of themselves. Emergence 
in perception does not imply that the mind supervenes on 
the brain, but rather it indicates that the neurophysiologi- 
cal processes involved in perception exhibit the kind of 
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holistic emergence seen in the soap bubble, where a multi- 
tude of tiny forces act together simultaneously to produce 
a final perceptual state by way of a process that cannot be 
reduced to simple laws. 


7.2. Reification 


The Kanizsa figure (Kanizsa 1979), shown in Figure 4A, is 
one of the most familiar illusions introduced by Gestalt the- 
ory. In this figure the triangular configuration is not only 
recognized as being present in the image, but that triangle 
is filled-in perceptually, producing visual edges in places 
where no edges are present in the input; and those edges in 
turn are observed to bound a uniform triangular region that 
is brighter than the white background of the figure. Idesawa 
(1991) and Tse (1999a; 1999b) extended this concept with 
a set of even more sophisticated illusions, including those 
shown in Figures 4B—D, in which the illusory percept takes 
the form of a three-dimensional volume. These figures 
demonstrate that the visual system performs a perceptual 
reification, a filling-in of a more complete and explicit per- 
ceptual entity based on a less complete visual input. Reifi- 
cation is a general principle of perceptual processing, of 
which boundary completion and surface filling-in are more 
specific computational components. The identification of 
this generative aspect of perception is one of the most sig- 
nificant contributions of Gestalt theory. 


7.3. Multistability 


A familiar example of multistability in perception is seen in 
the Necker cube, shown in Figure 5A. Prolonged viewing 
of this stimulus results in spontaneous reversals, in which 
the entire percept is observed to invert in depth. Figure 5B 
shows how large regions of the percept invert coherently in 
bistable fashion. Even more compelling examples of multi- 
stability are seen in surrealistic paintings by Salvator Dali 
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Figure 4. A. The Kanizsa triangle. B. Tse’s volumetric worm. C. 
Idesawa’s spiky sphere. D. Tse’s “sea monster.” 
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Figure 5. A. The Necker cube demonstrates multistability in 
perception. B. This figure shows how large regions of the percept 
flip coherently between perceptual states. 


and etchings by Escher, in which large and complex regions 
of an image are seen to invert perceptually, losing all re- 
semblance to their former appearance (Attneave 1971). 
The significance for theories of visual processing is that 
perception cannot be considered as simply a feed-forward 
processing performed on the visual input to produce a per- 
ceptual output, as it is most often characterized in compu- 
tational models of vision, but rather perception must in- 
volve some kind of dynamic process whose stable states 
represent the final percept. 


7.4, Invariance 


A central focus of Gestalt theory is the issue of invariance — 
how an object, like a square or a triangle, can be recognized 
regardless of its rotation, translation, or scale, or whatever 
its contrast polarity against the background, or whether it is 
depicted in solid or outline form, or whether it is defined in 
terms of texture, motion, or binocular disparity. This in- 
variance is not restricted to the two-dimensional plane but 
is also observed through rotation in depth, and even in in- 
variance to perspective transformation. For example, the 
rectangular shape of a tabletop is recognized even when its 
retinal projection is in the form of a trapezoid due to per- 
spective, and yet when we view the tabletop from any par- 
ticular perspective we can still identify the exact contours 
in the visual field that correspond to the boundaries of the 
perceived table, to the highest resolution of the visual sys- 
tem. The ease with which these invariances are handled in 
biological vision suggests that invariance is fundamental to 
the visual representation. 

Our failure to find a neurophysiological explanation for 
Gestalt phenomena does not suggest that no such explana- 
tion exists, only that we must be looking for it in the wrong 
places. The enigmatic nature of Gestalt phenomena high- 
lights the importance of the search for a computational 
mechanism that exhibits these same properties. In the next 
section, I present a model that demonstrates how these 
Gestalt principles can be expressed in a computational 
model that is isomorphic with the subjective experience of 
vision. 


8. The computational mechanism of perception 
The basic function of visual perception can be described as 


the transformation from a two-dimensional retinal image, 
or a pair of images in the binocular case, to a solid three-di- 
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Figure 6. A. A line drawing stimulates B, a volumetric spatial 
percept with an explicit depth value at every point on every visible 
surface, and an amodal percept of hidden rear surfaces. C. The 
central Y vertex from panel A, which tends to be perceived as a 
corner in depth. D. A dynamic rod-and-rail model of the emer- 
gence of the depth percept in panel C by relaxation of local con- 
straints. 


mensional percept. Figure 6A depicts a two-dimensional 
stimulus that produces a three-dimensional percept of a 
solid cube complete in three dimensions. For simplicity, a 
simple line drawing is depicted in the figure, but the argu- 
ment applies more appropriately to a view of a real cube ob- 
served in the world. Every point on every visible surface of 
the percept is experienced at a specific location in depth, 
and each of those surfaces is experienced as a planar con- 
tinuum, with a specific three-dimensional slope in depth. 
The information in this perceptual experience can there- 
fore be expressed as a three-dimensional model, as sug- 
gested in Figure 6B, constructed on the basis of the input 
image in Figure 6A. 

The transformation from a two-dimensional image space 
to a three-dimensional perceptual space is known as the in- 
verse optics problem because the intent is to reverse the 
optical projection in the eye, in which three-dimensional 
information from the world is collapsed into a two-dimen- 
sional image. However, the inverse optics problem is un- 
derconstrained, for there are an infinite number of possible 
three-dimensional configurations that can give rise to the 
same two-dimensional projection. How does the visual sys- 
tem select from this infinite range of possible percepts to 
produce the single perceptual interpretation observed phe- 
nomenally? The answer to this question is of central signif- 
icance to understanding the principles behind perception, 
for it reveals a computational strategy quite unlike anything 
devised by man, and certainly unlike the algorithmic deci- 
sion sequences embodied in the paradigm of digital com- 
putation. The transformation observed in visual perception 
gives us the clearest insight into the nature of this unique 
computational strategy. I propose that the principles of 
emergence, reification, and multistability are intimately in- 
volved in this reconstruction, and that in fact these Gestalt 
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properties are exactly the properties needed for the visual 
system to address the fundamental ambiguities inherent in 
reflected light imagery. 

The principle behind the perceptual transformation can 
be expressed in general terms as follows. For any given vi- 
sual input there is an infinite range of possible configura- 
tions of objects in the external world which could have given 
rise to that same stimulus. The configuration of the stimu- 
lus constrains the range of possible perceptual interpreta- 
tions to those that line up with the stimulus in the two di- 
mensions of the retinal image. Although each individual 
interpretation within that range is equally likely with re- 
spect to the stimulus, some of those perceptual alternatives 
are intrinsically more likely than others, in the sense that 
they are more typical of objects commonly found in the 
world. I propose that the perceptual representation has the 
property that the more likely structural configurations are 
also more stable in the perceptual representation, and 
therefore the procedure used by the visual system is to es- 
sentially construct or reify all possible interpretations of a 
visual stimulus in parallel, as constrained by the configura- 
tion of the input, and then to select from that range of pos- 
sible percepts the most stable perceptual configuration by 
a process of emergence. In other words, perception can be 
viewed as the computation of the intersection of two sets of 
constraints, which might be called extrinsic and intrinsic 
constraints. The extrinsic constraints are those determined 
by the visual stimulus, the intrinsic constraints are deter- 
mined by the structural stability of the percept. 

Arnheim (1969) presented an insightful analysis of this 
concept, which can be reformulated as follows. Consider 
(for simplicity) just the central Y vertex of Figure 6A de- 
picted in Figure 6C. Arnheim proposed that the extrinsic 
constraints of inverse optics can be expressed for this stim- 
ulus using a rod-and-rail analogy as shown in Figure 6D. 
The three rods, representing the three edges in the visual 
input, are constrained in two dimensions to the configura- 
tion seen in the input, but are free to slide in depth along 
the four rails. The rods must be elastic between their end- 
points, so that they can expand and contract in length. By 
sliding along the rails, the rods can take on any of the infi- 
nite three-dimensional configurations corresponding to the 
two-dimensional input of Figure 6C. For example, the final 
percept could theoretically range from a percept of a con- 
vex vertex protruding from the depth of the page to a con- 
cave vertex intruding into the depth of the page, with a con- 
tinuum of intermediate perceptual states between these 
limits. 

There are other possibilities beyond these, such as per- 
cepts where each of the three rods is at a different depth 
and therefore they do not meet in the middle of the stimu- 
lus. However, these alternative perceptual states are not all 
equally likely to be experienced. Hochberg and Brooks 
(1960) showed that the final percept is the one that exhibits 
the greatest simplicity, or prdégnanz. In the case of the ver- 
tex of Figure 6C the percept tends to appear as three rods 
whose ends coincide in depth at the center, and meet at a 
mutual right angle, defining either a concave or convex cor- 
ner. This reduces the infinite range of possible configura- 
tions to two discrete perceptual states. This constraint can 
be expressed emergently in the rod-and-rail model by join- 
ing the three rods flexibly at the central vertex, and in- 
stalling spring forces that tend to hold the three rods at mu- 
tual right angles at the vertex. With this mechanism in place 
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to define the intrinsic or structural constraints, the rod-and- 
rail model becomes a dynamic system that slides in depth 
along the rails, and this system is bistable between a con- 
cave and a convex right-angled percept, as observed phe- 
nomenally in Figure 6C. Although this model reveals the 
dynamic interaction between intrinsic and extrinsic con- 
straints, this particular analogy is hardwired to modeling the 
percept of the triangular vertex of Figure 6C. I will now de- 
velop a more general model that operates on this same dy- 
namic principle, but is designed to handle arbitrary input 
patterns. 


8.1. A Gestalt Bubble model 


For the perceptual representation, I propose (Lehar 2003b) 
a volumetric block or matrix of dynamic computational el- 
ements, as suggested in Figure 7A, each of which can exist 
in one of two states, transparent or opaque, with opaque- 
state units being active at all points in the volume of per- 
ceptual space where a colored surface is experienced. In 
other words, upon viewing a stimulus like that in Figure 6A, 
the perceptual representation of this stimulus is modeled as 
a three-dimensional pattern of opaque-state units embed- 
ded in the volume of the perceptual matrix in exactly the 
configuration observed in the subjective perceptual experi- 
ence when viewing Figure 6A — with opaque-state ele- 
ments at all points in the volumetric space that are within a 
perceived surface in three dimensions, as suggested in Fig- 
ure 6B. All other elements in the block are in the transpar- 
ent state to represent the experience of the spatial void 
within which perceived objects are perceived to be em- 
bedded. More generally, opaque-state elements should also 
encode the subjective dimensions of color (hue, intensity, 
and saturation), and intermediate states between transpar- 
ent and opaque would be required to account for the per- 
ception of semitransparent surfaces, although for now the 
discussion will be limited to two states and the monochro- 
matic case. The transformation of perception can now be 
defined as the turning on of the appropriate pattern of ele- 
ments in this volumetric representation in response to the 
visual input, in order to replicate the three-dimensional 
configuration of surfaces experienced in the subjective per- 
cept. 


8.2. Surface percept interpolation 


The perceived surfaces due to a stimulus like Figure 6A ap- 
pear to span the structure of the percept defined by the 
edges in the stimulus, somewhat like a milky bubble surface 
clinging to a cubical wire frame. Although the featureless 
portions of the stimulus between the visual edges offer no 
explicit visual information, a continuous surface is per- 
ceived within those regions, as well as across the white 
background behind the block figure, with a specific depth 
and surface orientation value encoded explicitly at each 
point in the percept. This three-dimensional surface inter- 
polation function can be expressed in the perceptual model 
by assigning to every element in the opaque state a surface 
orientation value in three dimensions, and by defining a dy- 
namic interaction between opaque-state units to fill in the 
region between them with a continuous surface percept. In 
order to express this process as an emergent one, the dy- 
namics of this surface interpolation function must be de- 
fined in terms of local fieldlike forces analogous to the lo- 
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Figure 7. A. The Gestalt Bubble model consisting of a block of 
dynamic local elements which can be in one of several states. B. 
The transparent state, no neighborhood interactions. C. The 
opaque coplanarity state which tends to complete smooth sur- 
faces. D. The opaque orthogonality state which tends to complete 
perceptual corners. E. The opaque occlusion state which tends to 
complete surface edges. 


cal forces of surface tension active at any point in a soap 
bubble. Figure 7B represents a point in the perceptual ma- 
trix in the transparent state, representing a percept of 
empty space at that location. Elements in this state do not 
project field-like forces to adjacent elements. Figure 7C de- 
picts an opaque-state unit representing a local portion of a 
perceived surface at a specific three-dimensional location 
and with a specific surface orientation. The planar field of 
this element, depicted somewhat like a planetary ring in 
Figure 7C, represents both the perceived surface repre- 
sented by this element, as well as a fieldlike influence prop- 
agated by that element to adjacent units. This planar field 
fades smoothly with distance from the center with a Gauss- 
ian function. The effect of this field is to recruit adjacent el- 
ements within that field of influence to take on a similar 
state — that is, to induce transparent state units to switch to 
the opaque state, and opaque-state units to rotate toward a 
similar surface orientation value. The final state and orien- 
tation taken on by any element is computed as a spatial av- 
erage or weighted sum of the states of neighboring units as 
communicated through their planar fields of influence, 
with the greatest influence from nearby opaque elements 
in the matrix. The influence is reciprocal between neigh- 
boring elements, thereby defining a circular relation as sug- 
gested by the principle of emergence. To prevent runaway 
positive feedback and uncontrolled propagation of surface 
signal, an inhibitory dynamic is also incorporated. This dy- 
namic suppresses surface formation out of the plane of the 
emergent surface by endowing the local field of each unit 
with an inhibitory field that suppresses the opaque state in 
neighboring elements in all directions outside of the plane 
of its local field. The mathematical specification of the lo- 
cal field of influence between opaque-state units is outlined 
in greater detail in the Appendix. However, the intent of the 
model is expressed more naturally in the global properties 


as described here, so the details of the local field influences 
are presented as only one possible implementation of the 
concept, provided in order to ground this somewhat nebu- 
lous idea in more concrete terms. 

The global properties of the system should be such that 
if the elements in the matrix were initially assigned ran- 
domly to either the transparent or opaque state, with ran- 
dom surface orientations for opaque-state units, the mutual 
fieldlike influences would tend to amplify any group of 
opaque-state elements whose planar fields happened to be 
aligned in an approximate plane; and as that plane of active 
units feeds back on its own activation, the orientations of its 
elements would conform ever closer to that of the plane, 
and elements outside of the plane would be suppressed to 
the transparent state. This would result in the emergence 
of a single plane of opaque-state units as a dynamic global 
pattern of activation embedded in the volume of the matrix, 
and that surface would be able to flex and stretch much like 
a bubble surface. Although, unlike a real bubble, this sur- 
face is defined not as a physical membrane but as a dynamic 
sheet of active elements embedded in the matrix. This vol- 
umetric surface interpolation function will now serve as the 
backdrop for an emergent reconstruction of the spatial per- 
cept around a three-dimensional skeleton or framework 
constructed on the basis of the visual edges in the scene. 


8.3. Local effects of a visual edge 


A visual edge can be perceived as an object in its own right, 
like a thin rod or wire surrounded by empty space. More of- 
ten, however, an edge is seen as a discontinuity in a surface, 
either as a corner or a fold, or perhaps as an occlusion edge 
like the outer perimeter of a flat figure viewed against a 
more distant background. The interaction between a visual 
edge and a perceived surface can therefore be modeled as 
follows: The two-dimensional edge from the retinal stimu- 
lus projects a different kind of field of influence into the 
depth dimension of the volumetric matrix, as suggested by 
the gray shading in Figure 7A, to represent the three-di- 
mensional locus of all possible edges that project to the two- 
dimensional edge in the image. In other words, this field ex- 
presses the inverse optics probability field or extrinsic 
constraint due to a single visual edge. Wherever this field 
intersects opaque-state elements in the volume of the ma- 
trix, it changes the shape of their local fields of influence 
from a coplanar interaction to an orthogonal, or corner, in- 
teraction as suggested by the local force field in Figure 7D. 
The corner of this field should align parallel to the visual 
edge but otherwise remain unconstrained in orientation ex- 
cept by interactions with adjacent opaque units. Visual 
edges can also denote occlusion, and so opaque-state ele- 
ments can also exist in an occlusion state, with a coplanarity 
interaction in one direction only, as suggested by the oc- 
clusion field in Figure 7E. Therefore, in the presence of a 
single visual edge, a local element in the opaque state 
should have an equal probability of changing into the or- 
thogonality or occlusion state, with the orthogonal or oc- 
clusion edge aligned parallel to the inducing visual edge. 
Elements in the orthogonal state tend to promote orthogo- 
nality in adjacent elements along the perceived comer, 
whereas elements in the occlusion state promote occlusion 
along that edge. In other words, an edge will tend to be per- 
ceived as a corner or occlusion percept along its entire 
length, although the whole edge may change state back and 
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Figure 8. Several possible stable states of the Gestalt Bubble 
model in response to a single visual edge. 


forth as a unit in a multistable manner. (The Appendix pre- 
sents a more detailed mathematical description of how 
these orthogonality and occlusion fields might be defined.) 
The presence of the visual edge in Figure 7A tends to crease 
or break the perceived surface into one of the several pos- 
sible configurations shown in Figures 8A—D. The final con- 
figuration selected by the system would depend not only on 
the local image region depicted in Figure 8, but also on 
forces from adjacent regions of the image, in order to fuse 
the orthogonal or occlusion state elements seamlessly into 
nearby coplanar surface percepts. 


8.4. Global effects of configurations of edges 


Visual illusions like the Kanizsa figure shown in Figure 4A 
suggest that edges in a stimulus that are in a collinear con- 
figuration tend to link up in perceptual space to define a 
larger global edge connecting the local edges. This kind of 
collinear boundary completion is expressed in this model as 
a physical process analogous to the propagation of a crack 
or fold in a physical medium. A visual edge that fades grad- 
ually produces a crease in the perceptual medium which 
tends to propagate outward beyond the edge as suggested 
in Figure 9A. If two such edges are found in a collinear con- 
figuration, the perceptual surface will tend to crease or fold 
between them as suggested in Figure 9B. This tendency is 
accentuated if additional evidence from adjacent regions 
supports this configuration. This can be seen in Figure 9D 
where fading horizontal lines are seen to link up across the 
figure to create a percept of a folded surface in depth, 
which would otherwise appear as a regular hexagon as in 
Figure 9C. 

Gestalt theory emphasizes the significance of closure as 
a prominent factor in perceptual segmentation because an 
enclosed contour is seen to promote a figure/ground seg- 
regation (Koffka 1935, p. 178). For example, an outline 
square tends to be seen as a square surface in front of a 
background surface that is complete and continuous be- 
hind the square, as suggested in the perceptual model de- 
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Figure 9. A. Boundary completion in the Gestalt bubble model: 
A single line ending creates a crease in the perceptual surface. B. 
Two line endings generate a crease joining them. C. A regular 
hexagon figure transforms into D, a percept of a folded surface in 
depth, with the addition of suggestive lines, with the assistance of 


a global gestalt that is consistent with that perceptual interpreta- 
tion. 
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picted in Figure 10A. The problem is that closure is a 
gestaltqualitdt, a quality defined by a global configuration 
that is difficult to specify in terms of any local featural re- 
quirements, especially in the case of irregular or frag- 
mented contours as seen in Figure 10B. In this model, an 
enclosed contour breaks away a piece of the perceptual sur- 
face, completing the background amodally behind the oc- 
cluding foreground figure. In the presence of irregular or 
fragmented edges the influences of the individual edge 
fragments act collectively to break the perceptual surface 
along that contour as suggested in Figure 10C, like the 
breaking of a physical surface that is weakened along an ir- 
regular line of cracks or holes. The final scission of figure 
from ground is therefore driven not so much by the exact 
path of the individual irregular edges as by the global con- 
figuration of the emergent gestalt. 


8.5. Vertices and intersections 


In the case of vertices or intersections between visual 
edges, the different edges interact with one another, favor- 
ing the percept of a single vertex at that point. For exam- 
ple, the three edges defining the three-way Y vertex shown 
in Figure 6C promote the percept of a single three-dimen- 
sional corner whose depth profile depends on whether the 
corner is perceived as convex or concave. In the case of Fig- 
ure 6A, the cubical percept constrains the central Y vertex 
as a convex rather than a concave trihedral percept. I pro- 
pose that this dynamic behavior can be implemented using 
the same kinds of local field forces described in the Ap- 
pendix to promote mutually orthogonal completion in three 
dimensions, wherever visual edges meet at an angle in two 
dimensions. Figure 11A depicts the three-dimensional in- 
fluence of the two-dimensional Y vertex when projected on 
the front face of the volumetric matrix. Each plane of this 
three-planed structure promotes the emergence of a cor- 
ner or occlusion percept at some depth within that plane. 
But the effects due to these individual edges are not inde- 
pendent. Consider first the vertical edge projecting from 
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Figure 10. A. The perception of closure and figure/ground seg- 
regation are explained in the Gestalt Bubble model exactly as per- 
ceived, in this case as a foreground square in front of a background 
surface that completes behind the square. B. Even irregular and 
fragmented surfaces produce a figure/ground segregation. C. The 
perceived boundary of the fragmented figure follows the global 
emergent gestalt rather than the exact path of individual edges. 


the bottom of the vertex. By itself, this edge might produce 
a folded percept as suggested in Figure 11B, which could 
occur through a range of depths and a variety of orienta- 
tions in depth, as well as in concave or convex form. But the 
two angled planes of this percept each intersect the other 
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Figure 11. A. The three-dimensional field of influence due to a 
two-dimensional Y vertex projected into the depth dimension of 
the volumetric matrix. B. Each field of influence, for example the 
one due to the vertical edge, stimulates a folded surface percept. 
The folded surface intersects the other fields of influence due to 
the other two edges, thereby tending to produce a percept of a sin- 
gle corner percept. C. One of many possible emergent surface 
percepts in response to that stimulus, in the form of a convex tri- 
hedral surface percept. D. The percept can also be of a concave 
trihedral corner, as seen sometimes at the center in this bistable 
figure. 


two fields of influence due to the other two edges of the 
stimulus, as suggested in Figure 11B, thus favoring the 
emergence of those edges’ perceptual folds at that same 
depth, and resulting in a single trihedral percept at some 
depth in the volumetric matrix, as suggested in Figure 11C. 
Any dimension of this percept that is not explicitly specified 
or constrained by the visual input remains unconstrained. 
In other words, the trihedral percept is embedded in the 
volumetric matrix in such a way that its three component 
comer percepts are free to slide inward or outward in 
depth, to rotate through a small range of angles, and to flip 
in a bistable manner between a convex and a concave tri- 
hedral configuration. The model now expresses the multi- 
stability of the rod-and-rail analogy shown in Figure 6D but 
in a more generalized form that is no longer hardwired to 
the Y-vertex input shown in Figure 6C; it can accommodate 
any arbitrary configuration of lines in the input image. A lo- 
cal visual feature like an isolated Y vertex generally exhibits 
a larger number of stable states, whereas in the context of 
adjacent features the number of stable solutions is often di- 
minished. This explains why the cubical percept of Figure 
6A is stable, but its central Y vertex alone, as shown in Fig- 
ure 6C, is bistable. The fundamental multistability of Fig- 
ure 6A can be revealed by the addition of a different spatial 
context, as depicted in Figure 11D. 


8.6. Perspective cues 


Perspective cues offer another example of a computation 
that is inordinately complicated in most models. In a fully 
reified spatial model, however, perspective can be com- 
puted relatively easily with only a small change in the geom- 
etry of the model. Figure 12A shows a trapezoid stimulus, 
which has a tendency to be perceived in depth, with the 
shorter top side being perceived as of the same length as 
the longer base but apparently diminished by perspective. 
Arnheim (1969) suggested a simple distortion to the volu- 
metric model to account for this phenomenon, which can 
be reformulated as follows. The height and width of the vol- 
umetric matrix are diminished as a function of depth, as 
suggested in Figure 12B, transforming the block shape into 
a truncated pyramid that tapers in depth. The vertical and 
horizontal dimensions represented by that space, however, 
are not diminished; in other words, the larger front face and 
the smaller rear face of the volumetric structure represent 
equal areas in perceived space by unequal areas in repre- 
sentational space, as suggested by the converging gridlines 
in the figure. All of the spatial interactions described above 
(e.g., the collinear propagation of comer and occlusion per- 
cepts) would be similarly distorted in this space. Even the 
angular measure of orthogonality is distorted somewhat by 
this transformation. The perceived cube depicted in the 
solid volume of Figure 12B is metrically shrunk in height 
and width as a function of depth, but because this shrink- 
ing is in the same proportion as the shrinking of the space 
itself, the depicted irregular cube represents a percept of a 
regular cube with equal sides and orthogonal faces. 

The propagation of the field of influence in depth due to 
a two-dimensional visual input, however, does not shrink 
with depth. A projection of the trapezoid of Figure 12A 
would occur in this model as depicted in Figure 12C, pro- 
jecting the trapezoidal form backward in parallel, indepen- 
dent of the convergence of the space around it. The shaded 
surfaces in Figure 12C therefore represent the loci of all 
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Figure 12. A. A trapezoidal stimulus that tends to be perceived 
as a rectangle viewed in perspective. B. The perspective modified 
spatial representation whose dimensions are shrunken in height 
and breadth as a function of depth. C. The parallel projection of 
a field of influence into depth of the two-dimensional trapezoidal 
stimulus. D. Several possible perceptual interpretations of the 
trapezoidal stimulus, one of which (depicted in black outline) rep- 
resents a regular rectangle viewed in perspective, because the 
convergence of its sides exactly matches the convergence of the 
space itself. 





possible spatial interpretations of the two-dimensional 
trapezoid stimulus of Figure 12A, or the extrinsic con- 
straints for the spatial percept due to this stimulus. One 
possible perceptual interpretation is of a trapezoid parallel 
to the plane of the page, which can be perceived to be ei- 
ther nearer or farther in depth, but because the scale size 
shrinks as a function of depth, the percept will be experi- 
enced as larger in absolute size (as measured against the 
shrunken spatial scale) when perceived as farther away and 
as smaller in absolute size (as measured against the ex- 
panded scale) when perceived to be closer in depth. This 
corresponds to the phenomenon known as Emmert’s Law 
(Coren et al. 1994), whereby a retinal afterimage appears 
larger when viewed against a distant background than when 
viewed against a nearer background. 

There are also an infinite number of alternative percep- 
tual interpretations of the trapezoidal stimulus, some of 
which are depicted by the dark shaded lines in Figure 12D. 
Most of these alternative percepts are geometrically irreg- 
ular, representing figures with unequal sides and odd an- 
gles. But of all the possibilities, there is one special case, de- 
picted by the bold outline in Figure 12D, in which the 
convergence of the sides of the perceived form happens to 
coincide exactly with the convergence of the space itself. In 
other words, this particular percept represents a regular 
rectangle viewed in perspective, with parallel sides and 
right-angled corners, whose nearer (bottom) and farther 
(top) horizontal edges are the same length in the distorted 
perceptual space. Although this rectangular percept repre- 
sents the most stable interpretation, other possible inter- 
pretations might be suggested by different contexts. The 
most significant feature of this concept of perceptual pro- 
cessing is that the result of the computation is expressed not 
in the form of abstract variables encoding the depth and 
slope of the perceived rectangle, but in the form of an ex- 
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plicit three-dimensional replica of the surface as it is per- 
ceived to exist in the world. 


8.7. Bounding the representation 


An explicit volumetric representation of perceived space as 
proposed here must necessarily be bounded in some way in 
order to allow a finite representational space to map to the 
infinity of external space, as suggested in Figure 2 (in sect. 
6.3). The nonlinear compression of the depth dimension 
observed in phenomenal space can be modeled mathemat- 
ically with a vergence measure, which maps the infinity of 
Euclidean distance into a finite bounded range, as sug- 
gested in Figure 13A. This produces a representation rem- 
iniscent of museum dioramas, like the one depicted in Fig- 
ure 13B, where objects in the foreground are represented 
in full depth and the depth dimension is increasingly com- 
pressed with distance from the viewer, eventually collaps- 
ing into a flat plane corresponding to the background. This 
vergence measure is presented here merely as a nonlinear 
compression of depth in a monocular spatial representa- 
tion, as opposed to a real vergence value measured in a 
binocular system, although this system could of course 
serve both purposes in biological vision. Assuming unit sep- 
aration between the eyes in a binocular system, this com- 
pression is defined by the equation 


v = 2 atan(1/2r), 


where v is the vergence measure of depth and r is the Eu- 
clidean range, or distance in depth. Because vergence is 
large at short range and smaller at long range, it is actually 
the “a-complement” vergence measure p that is used in the 
representation, where p = (a7 — v) and p ranges from 0 at 
r = Oto watr = infinity. 

What does this kind of compression mean in an isomor- 





Figure 13. A. A vergence representation maps infinite distance 
into a finite range. B. This produces a mapping reminiscent of a 
museum diorama. C. The compressed reference grid in this com- 
pressed space defines intervals that are perceived to be of uniform 
size. 
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Figure 14. A. An azimuth/elevation/vergence representation 
maps the infinity of three-dimensional Euclidean space into a fi- 
nite perceptual space. B. The deformation of the infinite Carte- 
sian grid caused by the perspective transformation of the azimuth/ 
elevation/vergence representation. C. A view of a man walking 
down a road represented in the perspective distorted space. D. A 
section of the spherical space depicted in the same format as the 
perspective space shown in Figure 12. 


phic representation? If the perceptual frame of reference 
is compressed along with the objects in that space, then the 
compression need not be perceptually apparent. Figure 
13C depicts this kind of compressed reference grid. The 
unequal intervals between adjacent depth gridlines define 
intervals that are perceived to be of equal length, so the flat- 
tened cubes defined by the distorted grid would appear 
perceptually as regular cubes, of equal height, breadth, and 
depth. This compression of the reference grid to match the 
compression of space would, in a mathematical system with 
infinite resolution, completely conceal the compression 
from the percipient. In a real physical implementation 
there are two effects of this compression that would remain 
apparent perceptually, due to the fact that the spatial ma- 
trix itself would have to have a finite perceptual resolution. 
The resolution of depth within this space is reduced as a 
function of depth, and beyond a certain limiting depth, all 
objects are perceived to be flattened into two dimensions, 
with zero extent in depth. This phenomenon is observed 
perceptually, where the sun, moon, and distant mountains 
appear as if they are pasted against the flat dome of the sky. 

The other two dimensions of space can also be bounded 
by converting the x and y of Euclidean space into azimuth 
and elevation angles, « and 8, producing an angle/angle/ 
vergence representation, as shown in Figure 14A. Mathe- 
matically, this transformation converts the point P(a,8,r) in 
polar coordinates to point Q(a,B,p) in this bounded spher- 
ical representation. In other words, azimuth and elevation 
angles are preserved by this transformation, and the radial 
distance in depth r is compressed to the vergence repre- 
sentation p as described above. This spherical coordinate 
system has the ecological advantage that the space near the 
body is represented at the highest spatial resolution, 


whereas the less important more distant parts of space are 
represented at lower resolution. All depths beyond a cer- 
tain radial distance are mapped to the surface of the repre- 
sentation which corresponds to perceptual infinity. 

The mathematical form of this distortion is depicted in 
Figure 14B, where the distorted grid depicts the perceptual 
representation of an infinite Cartesian grid with horizontal 
and vertical gridlines spaced at equal intervals. This geo- 
metrical transformation from the infinite Cartesian grid ac- 
tually represents a unique kind of perspective transforma- 
tion on the Cartesian grid, with the transformed space 
appearing as a perspective view of a Cartesian grid when 
viewed from inside, with all parallel lines converging to a 
point in opposite directions. The significance of this obser- 
vation is that by mapping space into a perspective-distorted 
grid, the distortion of perspective is removed, in the same 
way that plotting log data on a log plot removes the loga- 
rithmic component of the data. Figure 14C shows how this 
space would represent the perceptual experience of a man 
walking down a road. If the distorted reference grid of Fig- 
ure 14B is used to measure lines and distances in Figure 
14C, the bowed line of the road on which the man is walk- 
ing is aligned with the bowed reference grid and therefore 
is perceived to be straight. The distortion of straight lines 
into curves in the perceptual representation is not immedi- 
ately apparent to the percipient because they are perceived 
to be straight. However, in a global sense there are peculiar 
distortions apparent to the percipient which are caused by 
this deformation of Euclidean space, for although the sides 
of the road are perceived to be parallel, they are also per- 
ceived to meet at a point on the horizon. 

The fact that two lines can be perceived to be both 
straight and parallel and yet to converge to a point both in 
front of and behind the percipient indicates that our inter- 
nal representation itself must be curved. The proposed rep- 
resentation of space has exactly this property. Parallel lines 
do not extend to infinity but meet at a point beyond which 
they are no longer represented. Likewise, the vertical walls 
of the houses in Figure 14C bow outward, away from the 
observer, but in doing so they follow the curvature of the 
reference lines in the grid of Figure 14B and are therefore 
perceived as being both straight and vertical. Because 
curved lines in this spherical representation in fact repre- 
sent straight lines in external space, all of the spatial inter- 
actions discussed in the previous section, including the 
coplanar interactions and the collinear creasing of per- 
ceived surfaces, must follow the grain or curvature of 
collinearity defined within this distorted coordinate system. 
The distance scale encoded in the grid of Figure 14B re- 
places the regularly spaced Cartesian grid by a nonlinear 
collapsing grid whose intervals are spaced ever closer as 
they approach perceptual infinity but nevertheless repre- 
sent equal intervals in external space. This nonlinear col- 
lapsing scale thereby provides an objective measure of dis- 
tance in the perspective-distorted perceptual world. The 
houses in Figure 14C, for example, would be perceived to 
be approximately the same size and depth, although the 
more-distant house is experienced at a lower perceptual 
resolution. 

Figure 14D depicts how a slice of Euclidean space of 
fixed height and width would appear in the perceptual 
sphere, extending to perceptual infinity in one direction, 
like a slice cut from the spherical representation of Figure 
14C. This slice is similar to the truncated pyramid shape 
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shown in Figure 12B, with the difference that the horizon- 
tal and vertical scales of representational space diminish in 
a nonlinear fashion as a function of distance in depth. In 
other words, the sides of the pyramid in Figure 14B con- 
verge in curves rather than in straight lines, and the pyra- 
mid is no longer truncated, but extends in depth all the way 
to the vanishing point at representational infinity. An input 
image is projected into this spherical space using the same 
principles as before. 


8.8. Brain anchoring 


One of the most disturbing properties of the phenomenal 
world for models of the perceptual mechanism involves the 
subjective impression that the phenomenal world rotates 
relative to our perceived head as our head turns relative to 
the world, and that objects in perception are observed to 
translate and rotate while maintaining their perceived 
structural integrity and recognized identity. This suggests 
that the internal representation of external objects and sur- 
faces is not anchored to the tissue of the brain, as suggested 
by current concepts of neural representation, but that per- 
ceptual structures are free to rotate and translate coher- 
ently relative to the neural substrate, as suggested in Kéh- 
ler’s field theory (Kohler & Held 1947). This issue of brain 
anchoring is so troublesome that it is often cited as a coun- 
terargument for an isomorphic representation because it is 
so difficult to conceive of the solid spatial percept of the sur- 
rounding world having to be reconstructed anew in all its 
rich spatial detail with every turn of the head (Gibson 1979; 
O’Regan 1992). However, an argument can be made for the 
adaptive value of a neural representation of the external 
world that could break free of the tissue of the sensory or 
cortical surface in order to lock on to the more meaningful 
coordinates of the external world, if only a plausible mech- 
anism could be conceived to achieve this useful property. 

Even in the absence of a neural model with the required 
properties, the invariance property can be encoded in a per- 
ceptual model. In the case of rotation invariance, this prop- 
erty can be quantified by proposing that the spatial struc- 
ture of a perceived object and its orientation are encoded 
as separable variables. This would allow the structural rep- 
resentation to be updated progressively from successive 
views of an object that is rotating through a range of orien- 
tations. The rotation invariance property does not mean, 
however, that the encoded form has no defined orientation, 
but rather that the perceived form is presented to con- 
sciousness at the orientation and rate of rotation that the ex- 
ternal object is currently perceived to possess. In other 
words, when viewing a rotating object, like a person doing 
a cartwheel or a skater spinning about her vertical axis, 
every part of that visual stimulus is used to update the cor- 
responding part of the internal percept even as that percept 
rotates within the perceptual manifold to remain in syn- 
chrony with the rotation of the external object. The per- 
ceptual model need not explain how this invariance is 
achieved neurophysiologically, it must merely express the 
invariance property computationally, regardless of the 
“neural plausibility” or computational efficiency of that cal- 
culation, for the perceptual model is more a quantitative 
description of the phenomenon than a theory of neuro- 
computation. 

The property of translation invariance can be similarly 
quantified in the representation by proposing that the 
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structural representation can be calculated from a stimulus 
that is translating across the sensory surface, to update a 
perceptual effigy that translates with respect to the repre- 
sentational manifold, while maintaining its structural in- 
tegrity. This accounts for the structural constancy of the 
perceived world as it scrolls past a percipient walking 
through a scene, with each element of that scene following 
the proper curved perspective lines as depicted in Figure 
2, expanding outward from a point up ahead, and collaps- 
ing back to a point behind, as would be seen in a cartoon 
movie rendition of Figure 2. 

The fundamental invariance of such a representation of- 
fers an explanation for another property of visual percep- 
tion, the way that individual impressions left by each visual 
saccade are observed to appear phenomenally at the ap- 
propriate location within the global framework of visual 
space depending on the direction of gaze. This property can 
be quantified in the perceptual model as follows. The two- 
dimensional image from the spherical surface of the retina 
is copied onto a spherical surface in front of the eyeball of 
the perceptual effigy, from whence the image is projected 
radially outward in an expanding cone into the depth di- 
mension of the internal perceptual world, as suggested in 
Figure 15, as an inverse analog of the cone of light received 
from the world by the eye. Eye, head, and body orientation 
relative to the external world are taken into account in or- 
der to direct the visual projection of the retinal image into 
the appropriate sector of perceived space, as determined 
from proprioceptive and kinesthetic sensations, in order to 
update the image of the body configuration relative to ex- 
ternal space. The percept of the surrounding environment 
therefore serves as a kind of three-dimensional frame 


\ 


Figure 15. The image from the retina is projected into the per- 
ceptual sphere from the center outward in the direction of gaze, 
as an inverse analog of the cone of light that enters the eye in the 
external world, taking into account eye, head, and body orienta- 
tion in order to update the appropriate portion of perceptual 
space. 
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buffer expressed in global coordinates. This frame buffer 
accumulates the information gathered in successive visual 
saccades and maintains an image of that external environ- 
ment in the proper orientation relative to a spatial model of 
the body, compensating for body rotations or translations 
through the world. Portions of the environment that have 
not been recently updated gradually fade from perceptual 
memory, which is why it is easy to bump one’s head after 
bending for some time under an overhanging shelf, or why 
it is possible to advance only a few steps safely after closing 
one’s eyes while walking. 


9. Discussion 


The picture of visual processing revealed by the phenome- 
nological approach is radically different from the picture re- 
vealed by neurophysiological studies. In fact, the computa- 
tional transformations observed phenomenologically are 
implausible in terms of contemporary concepts of neuro- 
computation and even in terms of computer algorithms. 
However, the history of psychology is replete with examples 
of plausibility arguments based on the limited technology 
of the time, arguments that were later invalidated by the 
emergence of new technologies. The outstanding achieve- 
ments of modern technology, especially in the field of in- 
formation processing systems, might seem to justify our 
confidence in judging the plausibility of proposed process- 
ing algorithms. And yet, despite the remarkable capabilities 
of modern computers, certain classes of problems appear 
to be fundamentally beyond the capacity of the digital com- 
puter. In fact, the very problems that are most difficult for 
computers to address, such as extraction of spatial structure 
from a visual scene, especially in the presence of attached 
shadows, cast shadows, specular reflections, occlusions, and 
perspective distortions, as well as the problems of naviga- 
tion in a natural environment, are problems that are rou- 
tinely handled by biological vision systems, even those of 
simpler animals. On the other hand, the kinds of problems 
that are easily solved by computers, such as perfect recall 
of vast quantities of meaningless data, perfect memory over 
indefinite periods, detection of the tiniest variation in oth- 
erwise identical data, or exact repeatability of even the most 
complex computations, are the kinds of problems that are 
inordinately difficult for biological intelligence, even that of 
the most complex of animals. It is therefore safe to assume 
that the computational principles of biological vision are 
fundamentally different from those of digital computation, 
and plausibility arguments predicated on contemporary 
concepts of what is computable are not applicable to bio- 
logical vision. If we allow that our contemporary concepts 
of neurocomputation are so embryonic that they should not 
restrict our observations of the phenomenal properties of 
perception, the evidence for a Gestalt Bubble model of per- 
ceptual processing becomes overwhelming. 

The phenomena of hallucinations and dreams demon- 
strate that the mind is capable of generating complete spa- 
tial percepts of the world, including a percept of the body 
and the space around it (Revonsuo 1995). It is unlikely that 
this remarkable capacity is used only to create such illusory 
percepts. More likely, dreams and hallucinations reveal the 
capabilities of an imaging system that is normally driven by 
the sensory input, generating perceptual constructs that are 
coupled to external reality. 


Studies of mental imagery (Kosslyn 1980; 1994) have 
characterized the properties of this imaging capacity and 
have confirmed the three-dimensional nature of the en- 
coding and processing of mental imagery. Pinker (1980) 
showed that the scanning time between objects in a re- 
membered three-dimensional scene increases linearly with 
increasing distance between objects in three dimensions. 
Shepard and Metzler (1971) showed that the time for rota- 
tion of mental images is proportional to the angle through 
which they are rotated. Kosslyn (1975) showed that it takes 
time to expand the size of mental images, and that smaller 
mental images are more difficult to scrutinize. As unex- 
pected as these findings may seem for theorists of neural 
representation, they are perfectly consistent with the sub- 
jective experience of mental imagery. On the basis of these 
findings, Pinker (1988) derived a volumetric spatial 
medium to account for the observed properties of mental 
image manipulation which is very similar to the model pro- 
posed here, with a volumetric azimuth/elevation coordi- 
nate system that is addressable both in subjective viewer- 
centered and objective viewer-independent coordinates, 
and with a compressive depth scale. 

The phenomenon of hemi-neglect (Heilman & Watson 
1977: Heilman et al. 1985; Kolb & Whishaw 1996; McFie 
& Zangwill 1960) reveals the effects of damage to spatial 
representation, as the capacity to represent spatial percepts 
in one half of phenomenal space is destroyed. Hemi-ne- 
glect patients are not simply blind to objects to one side, but 
are blind to the very existence of a space in that direction as 
a potential holder of objects. For example, they typically eat 
food only from the right half of the plate, and express sur- 
prise at the unexpected appearance of more food when the 
plate is rotated 180 degrees. This condition persists even 
when patients are cognitively aware of their deficit (Sacks 
1985). Bisiach and Luzatti (1978) and Bisiach et al. (1981) 
showed how this condition can also impair mental imaging 
ability. They described a neglect patient who, when in- 
structed to recall a familiar scene viewed from a certain di- 
rection, could recall only objects from the right half of his 
remembered space. When instructed to mentally turn 
around and face in the opposite direction, the patient could 
then recall only objects from the other side of the scene, ob- 
jects that now fell in the right half of his mental image 
space. The condition of hemi-neglect therefore suggests 
damage to the left half of a three-dimensional imaging 
mechanism that is used both for perception and for the gen- 
eration of mental imagery. Note that hemi-neglect also in- 
cludes a neglect of the left side of the body, which is con- 
sistent with the fact that the body percept is included as an 
integral part of the perceptual representation. 

The condition of hemi-neglect initially caused a great stir 
in psychological circles because it appeared to be concrete 
evidence for an explicit spatial representation in the brain 
(Bisiach & Luzzatti 1978; Bisiach et al. 1981; Denny-Brown 
& Chambers 1958; de Renzi 1982). It is curious that half of 
phenomenal space should have to disappear for psycholo- 
gists to take account of its existence in the first place. But 
after the initial excitement, the naive realists quickly mar- 
shalled their defenses with an array of arguments that many 
believe disposed of the troublesome issue of hemi-neglect. 
Some argued that hemi-neglect is not a failure of spatial 
representation but rather an imbalance of attention, or “ori- 
enting response’; that is, half of phenomenal space does not 
actually disappear, but the neglect patient is merely in- 
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clined to ignore its presence (Heilman & Watson 1977; 
Heilman et al. 1985; Kinsbourne 1987; 1993). Even if this 
argument were valid, it would not account for the presence 
in visual consciousness of the spatial structure of the phe- 
nomenal world whenever it is not being ignored or ne- 
glected; it would merely offer a convenient escape clause to 
make neglect syndrome seem no more mysterious than nor- 
mal spatial perception. Others argue that the phenomenon 
of hemi-neglect fractionates to a number of distinct pat- 
terns of impairment (Vallar 1998, p. 88). For example, many 
neglect patients can describe the global gestalt of a figure, 
but when copying its local features, they leave out those on 
the left side (Marshall & Halligan 1995). Present accounts 
of the multiple forms of neglect refer to several spatial maps 
and their interaction (e.g., Ladavas et al. 1997). This high- 
lights a conflict between the phenomenal and neurophysi- 
ological evidence, the former presenting a unified spatial 
structure in visual experience and the latter suggesting dis- 
crete mechanisms in different cortical areas. To the naive 
realist this suggests that the spatial percept must be some- 
how illusory, which thereby supposedly relieves neuro- 
science from any obligation to account for its manifest 
properties. 

What is curious about the debate over neglect is the pas- 
sion that it engenders. The evidence presented by each side 
never seems to convince the opposition, because the debate 
is not really about neglect but about its implications for per- 
ceptual representation, and that issue is not so much a mat- 
ter of experimental evidence but of the interpretation of 
that evidence, or the foundational assumptions with which 
one comes to the debate in the first place. Whatever the 
physiological reality behind the phenomenon of hemi-ne- 
glect, the Gestalt Bubble model offers at least a concrete 
description of this otherwise paradoxical phenomenon. 

The idea that this spatial imaging system employs an ex- 
plicit volumetric spatial representation is suggested by the 
fact that disparity tuned cells have been found in the cortex 
(Barlow et al. 1967), as predicted by the Projection Field 
Theory of binocular vision (Boring 1933; Charnwood 1951; 
Julesz 1971; Kaufman 1974; Marr & Poggio 1976), which is 
itself a volumetric model. Psychophysical evidence for a 
volumetric representation comes from the fact that per- 
ceived objects in depth exhibit attraction and repulsion in 
depth (Mitchison 1993; Westheimer & Levi 1987), in a 
manner that is suggestive of a short-range attraction and 
longer range repulsion in depth, analogous to the center- 
surround processing in the retina. Brookes and Stevens 
(1989) discussed the analogy between brightness and depth 
perception; they showed that a number of brightness illu- 
sions attributed to such center-surround processing have 
corresponding illusions in depth. Similarly, Anstis and 
Howard (1978) demonstrated a Craik-O’Brien-Cornsweet 
illusion in depth (cf. Cornsweet 1970) by cutting the near 
surface of a block of wood with a depth profile matching the 
brightness cusp of the brightness illusion, resulting in an il- 
lusory percept of a difference in depth of the surfaces on ei- 
ther side of the cusp. As in the brightness illusion, the depth 
difference at the cusp appears to propagate a perceptual in- 
fluence out to the ends of the block, suggesting a spatial dif- 
fusion of depth percept between depth edges. 

The many manifestations of constancy in perception 
have always posed a serious challenge for theories of per- 
ception because they reveal that the percept exhibits prop- 
erties of the distal object rather than of the proximal stim- 
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ulus, or pattern of stimulation on the sensory surface. The 
Gestalt Bubble model explains this by the fact that the in- 
formation encoded in the internal perceptual representa- 
tion itself reflects the properties of the distal object rather 
than the proximal stimulus. Size constancy is explained by 
the fact that objects perceived to be more distant are rep- 
resented closer to the outer surface of the perceptual 
sphere, where the collapsing reference grid corrects for the 
shrinkage of the retinal image due to perspective. An object 
perceived to be receding in depth, therefore, is expected 
perceptually to shrink in retinal size along with the shrink- 
ing of the grid in depth, and, conversely, shrinking objects 
tend to be perceived as receding. Rock and Brosgole (1964) 
showed that perceptual grouping by proximity is deter- 
mined not by proximity in the two-dimensional retinal pro- 
jection of the figure, but rather by the three-dimensional 
perceptual interpretation. A similar finding was shown by 
Green and Odum (1986). Shape constancy is exemplified 
by the fact that a rectangle seen in perspective is not per- 
ceived as a trapezoid, as its retinal image would suggest. 
The Miiller-Lyer and Ponzo illusions are explained in sim- 
ilar fashion (Gillam 1971; 1980; Gregory 1963; Tausch 
1954), the converging lines in those figures suggesting a 
surface sloping in depth, so that features near the converg- 
ing ends are measured against a more compressed refer- 
ence grid than the corresponding feature near the diverg- 
ing ends of those lines. 

Several researchers have presented psychophysical evi- 
dence for a spatial interpolation in depth, which is difficult 
to account for except with a volumetric representation in 
which the interpolation is computed explicitly in depth (At- 
tneave 1982). Kellman et al. (1996) have demonstrated a 
coplanar completion of perceived surfaces in depth in a 
manner analogous to the collinear completion in the 
Kanizsa figure. Barrow and Tenenbaum (1981, p. 94 and 
Fig. 6.1) showed how a two-dimensional wire-frame outline 
held in front of a dynamic random noise pattern stimulates 
a three-dimensional surface percept spanning the outline 
like a soap film, and that the perceived surface undergoes a 
Necker reversal together with the reversal of the perimeter 
wire. Ware and Kennedy (1978) showed that a three-di- 
mensional rendition of the Ehrenstein illusion, constructed 
of a set of rods converging on a circular hole, creates a 
three-dimensional version of the illusion that is perceived 
as a spatial structure in depth, even when rotated out of the 
fronto-parallel plane, complete with a perception of bright- 
ness at the center of the figure. This illusory percept ap- 
pears to hang in space like a faintly glowing disk in depth, 
reminiscent of the neon color spreading phenomenon. A 
similar effect can be achieved with a three-dimensional 
rendition of the Kanizsa figure. If the Ehrenstein and 
Kanizsa figures are explained by spatial interpolation in 
models such as that of Grossberg and Mingolla (1985), then 
the corresponding three-dimensional versions of these illu- 
sions must involve a volumetric computational matrix to 
perform the interpolation in depth. 

Collett (1985) investigated the interaction between 
monocular and binocular perception using stereoscopically 
presented line drawings in which some features were pre- 
sented only monocularly — that is, their depth information 
is unspecified. Collett showed that such features tend to ap- 
pear perceptually at the same depth as adjacent binocularly 
specified features, as if under the influence of an attractive 
force in depth generated by the binocular feature. In am- 
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biguous cases the percept is often multistable, jumping 
back and forth in depth, especially when monocular per- 
spective cues conflict with the binocular disparity infor- 
mation. The perceived depth of the monocularly specified 
surfaces is measured psychophysically using a three-di- 
mensional disparity-specified cursor whose depth is ad- 
justed by the subject to match the depth of the perceived 
surface at that point. Subjects reported a curious interac- 
tion between the cursor and the perceived surface, which 
was observed to flex in depth toward the cursor at small dis- 
parity differences, in the manner of the attraction and re- 
pulsion in depth reported by Westheimer and Levi (1987). 
This dynamic influence is suggestive of a grouping by prox- 
imity mechanism, expressed as a fieldlike attraction be- 
tween perceived features in depth; the flexing of the per- 
ceived surface near the three-dimensional cursor, as well as 
the multistability in the presence of conflicting perspective 
and disparity cues, is suggestive of a Gestalt Bubble model. 

Carman and Welch (1992) employed a similar cursor to 
measure the perceived depth of three-dimensional illusory 
surfaces seen in Kanizsa figure stereograms, whose induc- 
ing edges are tilted in depth in a variety of configurations, 
as shown in Figure 16A. Note how the illusory surface com- 
pletes in depth by coplanar interpolation defining a smooth 
curving surface. The subjects in this experiment also re- 
ported a flexing of the perceived surface in depth near the 
disparity-defined cursor. Equally interesting is the porthole 
illusion seen in the reverse-disparity version of this figure, 
where the circular completion of the portholes generates an 
ambiguous unstable semitransparent percept at the center 
of the figure, which is characteristic of the Gestalt Bubble 
model. Kellman and Shipley (1991) and Idesawa (1991) re- 
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Figure 16. Perceptual interpolation in depth in illusory figure 
stereograms. Panel A was adapted from Carman et al. (1992), B 
from Kellman and Shipley (1991), and C from Idesawa (1991). 
Opposite disparity percepts are achieved by binocular fusion of ei- 
ther the first and second, or the second and third columns of the 
figure. 


ported the emergence of more complex illusory surfaces in 
depth, using similar illusory stereogram stimuli as shown in 
Figures 16B and 16C. It is difficult to deny the reality of a 
precise high-resolution spatial interpolation mechanism in 
the face of these compelling illusory percepts. Whatever 
the neurophysiological basis of these phenomena, the 
Gestalt Bubble model offers a mathematical framework for 
a precise description of the information encoded in these 
elaborate spatial percepts, independent of the confounding 
factor of neurophysiological considerations. 

The sophistication of the perceptual reification capacity 
is revealed by the apparent-motion phenomenon (Coren et 
al. 1994), which in its simplest form consists of two alter- 
nately flashing lights that generate a percept of a single light 
moving back and forth between the flashing stimuli. With 
more complex variations of the stimulus, the illusory per- 
cept is observed to change color or shape in midflight, to 
carry illusory contours, or to carry a texture region bounded 
by an illusory contour between the alternately flashing stim- 
uli (Coren et al. 1994). Most pertinent to the discussion of 
a spatial representation is the fact that the illusory percept 
is observed to make excursions into the third dimension 
when that produces a simpler percept. For example, if an 
obstacle is placed between the flashing stimuli so as to block 
the path between them, the percept is observed to pass ei- 
ther in front of or behind the obstacle in depth. Similarly, if 
the two flashing stimuli are in the shape of an angular fea- 
ture like < or >, the angle is observed to rotate in depth 
between the flashing stimuli, preserving a percept of a rigid 
rotation in depth, in preference to a morphological defor- 
mation in two dimensions. The fact that the percept transi- 
tions so readily into depth suggests the fundamental nature 
of the depth dimension for perception. 

Although apparent-motion effects reify whole percep- 
tual gestalts, the elements of this reification, such as the 
fieldlike diffusion of perceived surface properties, are seen 
in such diverse phenomena as the perceptual filling-in of 
the Kanizsa figure (Takeichi et al. 1992), the Craik-O’Brien- 
Cornsweet effect (Cornsweet 1970), the neon color spread- 
ing effect (Bressan 1993), the filling-in of the blind spot 
(Ramachandran 1992), color bleeding due to retinal stabi- 
lization (Heckenmuller 1965; Yarbus 1967), the motion 
capture effect (Ramachandran & Anstis 1986), and the 
aperture problem in motion perception (Movshon et al. 
1986). In all of these phenomena, a perceived surface prop- 
erty (brightness, transparency, color, motion, etc.) is ob- 
served to spread from a localized origin, not into a fuzzy ill- 
defined region but, rather, into a sharply bounded region 
containing a homogeneous perceptual quality; and this fill- 
ing-in occurs as readily in depth in a perspective view as in 
the frontoparallel plane. The time has come to recognize 
that these phenomena do not represent exceptional or spe- 
cial cases, nor are they illusory in the sense of lacking a neu- 
rophysiological counterpart. Rather, these phenomena re- 
veal a general principle of neurocomputation that is 
ubiquitous in biological vision. 

Evidence for the spherical nature of perceived space 
dates back to observations by Helmholtz (1925). A subject 
in a dark room is presented with a horizontal line of point- 
lights at eye level in the frontoparallel plane, and instructed 
to adjust their displacement in depth, one by one, until they 
are perceived to lie in a straight line in depth. The result is 
a line of lights that curves inward toward the observer, the 
amount of curvature being a function of the distance of the 
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line of lights from the observer. Helmholtz recognized this 
phenomenon as evidence of the non-Euclidean nature of 
perceived space. The Hillebrand-Blumenfeld alley experi- 
ments (Blumenfeld 1913; Hillebrand 1902) extended this 
work with different configurations of lights, and later math- 
ematical analysis of the results (Blank 1958; Luneburg 
1950) characterized the nature of perceived space as Rie- 
mannian with constant Gaussian curvature (see Foley 1978; 
Graham 1965; and Indow 1991 for a review). In other 
words, perceived space bows outward from the observer, 
with the greatest distortion observed proximal to the body, 
as suggested by the Gestalt Bubble model. Heelan (1983) 
presented a more modern formulation of the hyperbolic 
model of perceived space, and provided further supporting 
evidence from art and illusion. 

It is perhaps too early to say definitively whether the 
model presented here can be formulated to address all of 
the phenomena outlined above. What is becoming increas- 
ingly clear, however, is the inadequacy of the conventional 
feed-forward abstraction approach to account for these 
phenomena, and that novel and unconventional approaches 
to the problem should be given serious consideration. The 
general solution offered by the Gestalt Bubble model to all 
of these problems in perception is that the internal percep- 
tual representation encodes properties of the distal object 
rather than of the proximal stimulus, and that the compu- 
tations of spatial perception are most easily performed in a 
fully spatial matrix in a manner consistent with the subjec- 
tive experience of perception. 


10. Conclusion 


I have presented an elaborate model of perception that in- 
corporates many of the concepts and principles introduced 
by the original Gestalt movement. Though the actual mech- 
anisms of the proposed model remain somewhat vague and 
poorly specified, a number of prominent aspects of visual 
experience which are generally ignored by other models are 
accounted for by this approach to modeling perception. 
These are summarized as follows. 

1. When we view a three-dimensional surface, our sub- 
jective experience of that surface simultaneously encodes 
every point on that surface in three dimensions at a high 
resolution; in other words, our subjective experience of the 
world around us is perceived not as a flattened “2-D 
sketch,” nor a nonspatial abstraction, but as a solid spatial 
world that appears to surround us in all directions. 

2. Volumes of empty space are perceived with the same 
geometrical fidelity as volumes of solid matter. 

3. Multiple transparent surfaces can be perceived si- 
multaneously as distinct spatial structures at high resolu- 
tion. 

4. The infinity of external space is perceived as a finite, 
but fully spatial, representation that appears near-Euclid- 
ean near the body but becomes progressively flattened with 
distance from the body, the entire percept being bounded 
by a spherical shell representing perceptual infinity. 

5. Parallel lines are perceived to meet at perceptual in- 
finity, but at the same time they are perceived as parallel 
and with uniform separation throughout their entire length. 

6. An illusory entity, like the Kanizsa figure or the ap- 
parent-motion illusion, is not experienced as a cognitive ab- 
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straction but is experienced perceptually as a solid spatial 
surface at high resolution, virtually indistinguishable from 
a real physical surface or object. 

7. The subjective reversal of a multistable percept is not 
experienced as a change in a cognitive interpretation or the 
flipping of a single cognitive variable but, rather, it is vividly 
experienced as an inversion of a perceptual data structure, 
changing the perceived depth of every point on the per- 
ceived structure. 

These phenomena are so immediately manifest in the 
subjective experience of perception that they need hardly 
be tested psychophysically. And yet, curiously, these most 
obvious properties of perception have been systematically 
ignored by neural modelers, even though the central sig- 
nificance of these phenomena was highlighted decades ago 
by the Gestaltists. There are two reasons why these promi- 
nent aspects of perception have been consistently ignored. 
The first results from the outstanding success of the single- 
cell recording technique, which shifted theoretical empha- 
sis from fieldlike theories of whole aspects of perception 
to pointlike theories of the elements of neural computa- 
tion. Like the classical Introspectionists, who refused to ac- 
knowledge perceptual experiences that were inconsistent 
with their preconceived notions of sensory representation, 
the Neuroreductionists of today refuse to consider aspects 
of perception that are inconsistent with current theories of 
neural computation, and some of them are even prepared 
to deny consciousness itself in a heroic attempt to save the 
sinking paradigm. 

There is another factor that has made it possible to ignore 
these most salient aspects of perception, which is that per- 
ceptual entities, such as the solid volumes and empty spaces 
we perceive around us, are easily confused with real objects 
and spaces in the objective external world. The illusion of 
perception is so compelling that we mistake the percept of 
the world for the real world itself. And yet this naive realist 
view that we can somehow perceive the world directly is in- 
consistent with the physics of perception. If perception is a 
consequence of neural processing of the sensory input, a 
percept cannot in principle escape the confines of our head 
to appear in the world around us, any more than a compu- 
tation in a digital computer can escape the confines of the 
computer. Therefore, we cannot in principle have direct ex- 
perience of objects in the world itself but only of the inter- 
nal effigies of those objects generated by mental processes. 
The world we see around us can only be an elaborate, 
though very compelling, illusion, which must in reality cor- 
respond to perceptual data structures and processes occur- 
ring actually within our own heads. As soon as we examine 
the world we see around us, not as a physical scientist ob- 
serving the physical world, but as a perceptual scientist ob- 
serving a rich and complex internal percept, only then does 
the rich spatial nature of perceptual processing become im- 
mediately apparent. It was this central insight into the illu- 
sion of consciousness that formed the key inspiration of the 
Gestalt movement, from which all of their other ideas were 
developed. The central message of Gestalt theory is that the 
primary function of perceptual processing is the generation 
of a miniature, virtual-reality replica of the external world 
inside our head, and that the world we see around us is not 
the real external world but is exactly that miniature internal 
replica (Lehar 2003b). It is only in this context that the elab- 
orate model presented here begins to seem plausible. 
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APPENDIX 


A1. The coplanarity field 


The mathematical form of the coplanarity interaction field can be 
described as follows. Consider the field strength F due to an ele- 
ment in the opaque state at some point in the volume of the spa- 
tial matrix, with a certain surface orientation, depicted in Figure 
A1, panel A, as a vector, representing the normal to the surface en- 
coded by that element. The strength of the field F should peak 
within the plane at right angles to this normal vector (depicted as 
a circle in panel A) as defined in polar coordinates by the function 
F., = sin(a), where a is the angle between the surface normal and 
some point in the field, that ranges from zero, parallel to the nor- 
mal vector, to 77, in the opposite direction. The sine function peaks 
at a = 1/2, as shown in panel B, producing an equatorial belt 
around the normal vector as suggested schematically in cross sec- 
tion in panel C, where the gray shading represents the strength of 
the field. The strength of the field should actually decay with dis- 
tance from the element, for example with an exponential decay 
function, as defined by the equation F,. = e~ sin(a) as shown 
in panel D, where r is the radial distance from the element. This 
produces a fading equatorial band, as suggested schematically in 
cross section in panel E. The equatorial belt of the function de- 
scribed so far would be rather fat, resulting in a lax or fuzzy copla- 
narity constraint, but the constraint can be stiffened by raising the 
sine to some positive power PB producing the equation F oa e 2 
sin(a)”, which will produce a sharper peak in the function as 
shown in panel F, producing a sharper in-plane field depicted 
schematically in cross section in panel G. To control runaway pos- 
itive feedback and suppress the uncontrolled proliferation of sur- 
faces, the field function should be normalized, which will project 
inhibition in directions outside the equatorial plane. This can be 
achieved with the equation F,. = e~'? 2 sin(a)’ — 1, which has 
the effect of shifting the equatorial function halfway into the neg- 
ative region, as shown in panel H, thereby producing the field sug- 
gested in cross section in panel I. 

The field described so far is unoriented — that is, it has a mag- 
nitude but no direction at any sample point (r,a). What is actually 
required is a field with a direction; such a field would have maxi- 
mal influence on adjacent elements that are oriented parallel to it, 
elements that are coplanar with it in both position and orientation. 
We can describe this orientation of the field with the parameter 0, 
which represents the orientation at which the field F is sampled, 








Figure Al. 





expressed as an angle relative to the normal vector. In other words, 
the strength of the influence F exerted on an adjacent element lo- 
cated at a point (r,@) varies with the deviation 0 of that element 
from the direction parallel to the normal vector, as shown in Fig- 
ure A2, such that the maximal influence is felt when the two ele- 
ments are parallel (i.e., when @ = 0, as in Fig. A2, panel A) and 
falls off smoothly as the other element’s orientation deviates from 
that orientation, as in Figure A2, panels B and C. 

This can be expressed with a cosine function, such that the in- 
fluence F of an element on another element in a direction « and 
at separation r from the first element, and with a relative orienta- 


tion 6, would be defined by 


F 49 =e 7 [2 sin(a)? — 1] | cos(6)® |. (Eq. 1) 


This cosine function allows the coplanar influence to propagate 
to near-coplanar orientations, thereby allowing surface comple- 
tion to occur around smoothly curving surfaces. The tolerance to 
such curvature can also be varied parametrically by raising the co- 
sine function to a positive power Q, as shown in equation 1. So the 
in-plane stiffness of the coplanarity constraint is adjusted by pa- 
rameter P, and the angular stiffness is adjusted by parameter Q. 
The absolute value on the cosine function in equation | allows in- 
teraction between elements when 6 is between 1/2 and 7. 


A2. The occlusion field 


The orthogonality and occlusion fields have one less dimension of 
symmetry than does the coplanarity field, and therefore they are 
defined with reference to two vectors through each element at 
right angles to each other, as shown in Figure A3, panel A. For the 
orthogonality field, these vectors represent the surface normals to 
the two orthogonal planes of the corner; for the occlusion field one 
vector is a surface normal and the other vector points within that 
plane in a direction orthogonal to the occlusion edge. The occlu- 
sion field G around the local element is defined in polar coordi- 
nates from these two vector directions, using the angles a and B 
respectively, as shown in Figure A3, panel A. 

The plane of the first surface is defined as for the coplanarity 
field, with the equation Gg, = e~' sin(a)’. For the occlusion 
field this planar function should be split in two, as shown in panel 
B, to produce a positive half and a negative half, so that this field 
will promote surface completion in one direction only and will ac- 
tually suppress surface completion in the negative half of the field. 


i: _ 
sini) — | ib 


Progressive construction of the equation for the coplanarity field from one element to another, as described in the text. 
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Figure A2. Orientation of the field of influence between one element and another. For an element located at polar coordinates (1,8), 
the influence varies as a cosine function of 0, the angle between the normal vectors of the two interacting elements. 


This can be achieved by multiplying the above equation by the 
sign (plus or minus, designated by the function sgn( )) of a cosine 
on the orthogonal vector; that is, Cae e~' sin(a)” sen(cos(B)). 
Because of the negative half-field in this function, there is no need 
to normalize the equation. However, the oriented component of 
the field can be added as before, resulting in the equation 


Gusea = e '? [sin(w)? sgn(cos(B))] | cos()® |. (Eq. 2) 
Again, the maximal influence will be experienced when the two 
elements are parallel in orientation, when @ = 0. As before, the 
orientation cosine function is raised to the positive power Q to al- 
low parametric adjustment of the stiffness of the coplanarity con- 
straint. 


A3. The orthogonality field 


The orthogonality field H can be developed in a similar manner, 
beginning with the planar function divided into positive and neg- 
ative half-fields — that is, with the equation Hyg, = e~™ sin(a)? 
sgn(cos(B)) — but then adding another similar plane from the or- 
thogonal surface normal, producing the equation Dae 
[sin(a)” sgn(cos(B)) + sin(B)” sgn(cos(«))]. This produces two or- 
thogonal planes, each with a negative half-field, as shown 
schematically in Figure A3, panel C. Finally, this equation must 
be modified to add the oriented component to the field, repre- 
sented by the vector 0, such that the maximal influence on an ad- 
jacent element will be experienced when that element is either 
within one positive half-plane and at one orientation or is within 
the other positive half-plane and at the orthogonal orientation. 
The final equation for the orthogonality field is therefore defined 


by 


— er? 


H e '? [sin(w)? sgn(cos(B)) | cos(8)° | + [sin(B)? 


aBro 
sgn(cos(a)) | cos(6)® |]. 


(Eq. 3) 





A4. Edge consistency and inconsistency constraints 


There is another aspect of the fieldlike interaction between ele- 
ments that remains to be defined. Both the orthogonal and occlu- 
sion states are promoted by appropriately aligned neighboring el- 
ements in the coplanar state. Orthogonal and occlusion elements 
should also feel the influence of neighboring elements in the or- 
thogonal and occlusion states, because a single edge should have 
a tendency to become either an orthogonal corner percept or an 
occlusion edge percept along its entire length. Therefore, orthog- 
onal or occlusion elements should promote like states and inhibit 
unlike states in adjacent elements along the same corner or edge. 
The interaction between like-state elements along the edge will 
be called the edge-consistency constraint, and the corresponding 
field of influence will be designated E; the complementary inter- 
action between unlike-state elements along the edge is called the 
edge-inconsistency constraint, and its corresponding edge-incon- 
sistency field will be designated I. These interactions are depicted 
schematically in Figure A4. 

The spatial direction along the edge can be defined by the prod- 
uct of the two sine functions, sin() sin(B), defining the orthogo- 
nal planes, denoting the zone of intersection of those two orthog- 
onal planes, as suggested in Figure A4, panel E. Again, this field 
can be sharpened by raising these sine functions to a positive 
power P, and it can be localized by applying the exponential decay 
function. The edge consistency constraint E therefore has the 
form E,,, = e ™ [sin(a)” sin(B)”]. As for the orientation of the 
edge-consistency field, this will depend now on two angles, @ and 
, representing the orientations of the two orthogonal vectors of 
the adjacent orthogonal or occlusion elements relative to the two 
normal vectors respectively. Both the edge-consistency and the 
edge-inconsistency fields, whether excitatory between like-state 
elements or inhibitory between unlike-state elements, should 
peak when both pairs of reference vectors are parallel to the nor- 
mal vectors of the central element — that is, when 6 and ¢ are both 


Figure A3. A. Polar coordinate reference vectors through each element. B. Occlusion field. C. Orthogonality field. 
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Figure A4. A, B. Edge consistency constraint as an excitatory influence between like-state elements along a corner or edge percept. 
C, D. Edge inconsistency constraint as an inhibitory influence between unlike-state elements along a corner or edge percept. E. The 
direction along the edge expressed as the intersection of the orthogonal planes defined by the sine functions on the two orthogonal vec- 


tors. 


equal to zero. The full equation for the edge-consistency field E 
would therefore be 


E e ' [sin(a)” sin(B)?] cos(8)2 cos()2, (Eq. 4) 


aBrod — 
where this equation is applied only to like-state edge or corner el- 
ements. The edge-inconsistency field I would be given by 


I e  [sin(a)? sin(B)?] cos()2 cos(b)2 (Eq. 5) 


aprod — 


applied only to unlike-state elements. The total influence R on an 
occlusion element therefore is calculated as the sum of the influ- 
ence of neighboring coplanar, orthogonal, and occlusion state el- 
ements as defined by 





R G + E I 


aBrod Brod (Eq. 6) 


aBrod — aBred? 


and the total influence S on an orthogonal state element is defined 


by 
S 





H + E 


aBrod — St aprod (Eq. 7) 


aBrod Li prec’ 


A5. Influence of the visual input 


A two-dimensional visual edge has an influence on the three-di- 
mensional interpretation of a scene because an edge is suggestive 
of either a corner or an occlusion at some orientation in three di- 
mensions whose two-dimensional projection coincides with that 
visual edge. This influence, however, is quite different from the 
local fieldlike influences described above, because the influence 
of a visual edge should penetrate the volumetric matrix with a pla- 
nar field of influence to all depths and should activate all local el- 
ements within the plane of influence that are consistent with that 
edge. Subsequent local interactions between those activated ele- 
ments serve to select which subset of them should finally repre- 
sent the three-dimensional percept corresponding to the two-di- 
mensional image. For example, a vertical edge as shown in Figure 
A5, panel A, would project a vertical plane of influence, as sug- 


gested by the light shading, into the depth dimension of the volu- 
metric matrix, where it stimulates the orthogonal and occlusion 
states that are consistent with that visual edge. It would stimulate 
corner and occlusion states at all angles about a vertical axis, as 





i visual ecloe 
plane of influcncn 


* fibod) axis 


COMer parce 


Vn. 


Figure A5. The influence of a visual edge, in this case a vertical 
edge, is to stimulate local elements in the occlusion or comer per- 
cept states at orientations about a vertical axis, panel A, or about 
a tilted axis, panel B, within the plane of influence of the edge. At 
equilibrium, panel C, a single unified percept emerges, in this 
case of a perceived corner at some depth and tilt in the volume of 
the matrix. 
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shown in panel A, where the circular disks represent different ori- 
entations of the positive half-fields of either corner or occlusion 
fields. 

However, a vertical edge would also be consistent with corners 
or occlusions about axes tilted relative to the image plane but 
within the plane of influence, as depicted in panel B. The same 
kind of stimulation would occur at every point within the plane of 
influence of the edge, although only one point is depicted in the 
figure. When all elements consistent with this vertical edge have 
been stimulated, the local fieldlike interactions between adjacent 
stimulated elements will tend to select one edge or corner at some 
depth and at some tilt, thereby suppressing alternative edge per- 
cepts at that two-dimensional location at different depths and at 
different tilts. At equilibrium, some arbitrary edge or corner per- 
cept will emerge within the plane of influence as suggested in 
panel C, which depicts only one such possible percept, and edge 
consistency interactions will promote like-state elements along 
that edge, producing a single emergent percept consistent with 
the visual edge. In the absence of additional influences, for exam- 
ple in the isolated local case depicted in panel C, the actual edge 
that emerges will be unstable; it could appear anywhere within the 
plane of influence of the visual edge through a range of tilt angles 
and could appear as either an occlusion or a corner edge. How- 
ever, when it does appear it propagates its own fieldlike influence 
into the volumetric matrix. In this example the corner percept 
would propagate a planar percept of two orthogonal surfaces that 
will expand into the volume of the matrix, as suggested by the ar- 
rows in panel C. The final percept therefore will be influenced by 
the global pattern of activity; that is, the final percept will con- 
struct a self-consistent perceptual whole whose individual parts 
reinforce one another by mutual activation by way of the local in- 
teraction fields, although that percept would remain unstable in 
all unconstrained dimensions. For example, the corner percept 
depicted in panel C would snake back and forth unstably within 
the plane of influence, rotate back and forth along its axis through 
a small angle, and flip alternately between the corner and occlu- 
sion states, unless the percept is stabilized by other features at 
more remote locations in the matrix. 
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Phenomenology is art, not psychological or 
neural science 


David A. Booth 
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Abstract: It is tough to relate visual perception or other achievements to 
physiological processing in the central nervous system. The diagrammatic, 
algebraic, and verbal pictures of how sights seem to Lehar do not advance 
understanding of how we manage to see what is in the world. There are 
well-known conceptual reasons why no such purely introspective ap- 
proach can be productive. 
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To see something is an achievement. That is to say, the claim to 
have performed correctly can be tested. Indeed, we can investi- 
gate how that task of visual recognition was successfully carried 
out. We can try to infer the information-transforming (cognitive) 
processes mediating the performance by varying what is visible 
and observing changes in response (i.e., doing psychophysics); this 
is an example of psychological science. 

The physical “engineering” of these processes of seeing can also 
be studied by varying the optical input, but this time observing 
what is projected onto the retina and activity in the central ner- 
vous system (CNS), from the rods and cones to V1 and beyond. 
Considerable progress has been made in relating cellular neuro- 
physiology to the psychophysics of elementary features of the vis- 
ible world. It is not so easy to get psychophysical evidence that dis- 
tinguishes between a cognitive process being in consciousness and 
transiently out of consciousness (Booth & Freeman 1993), al- 
though it is clear that some visual information processing never 
enters consciousness. When we cannot specify a mental process 
as conscious, there cannot be a theory of the neural basis of that 
process. Lehar’s complaint that neuroscience fails to explain visual 
consciousness is vacuous. 

Furthermore, what we know to be the case through use of our 
senses is a very different kettle of fish from the contents of con- 
sciousness, in the sense of how things seem to us while we discount 
our beliefs about how they actually are. By definition, how things 
seem cannot be checked against how things are. So the systema- 
tisation of expressions of subjective experience is an art form. 
Lehar’s diagrams, his field equations, and his verbal exposition are 
sophisticated elaborations of the sort of thing that I draw when I 
wake up and try to sketch the visual imagery that I was experi- 
encing as I woke. His and my graphic, algebraic, and verbal efforts 
cannot be wrong or right; they merely express how it appeared to 
be. 

Lehar says that his visual experience is holistic. I can empathise 
with that impression. Yet I also have visual experiences that are not 
holistic. I bet that he does too but chooses to ignore them. Any 
artist may do that, on the grounds that it would spoil the picture 
or detract from the story. However, that is aesthetics, not science. 

Lam not being positivistic. On the contrary, itis Lehar who com- 
mits the empiricists’ and rationalists’ epistemological fallacy of try- 
ing to build public knowledge on the basis of impressions or ideas 
that seem indubitable because they are private and so cannot be 
wrong — but then neither can they be right. Lehar writes: “These 
phenomena are so immediately manifest in the subjective experi- 
ence of perception that they need hardly be tested psychophysi- 
cally” (target article, sect. 10, para. 2). In words of one or two syl- 
lables: “What appears seeming to seem in seeing is so clearly clear 
that there is no need to test it against success at seeing.” 

Lehar’s paper is built on equivocation in use of the word “per- 
ception” between the objective achievement and subjective expe- 
rience. (The word “conscious” in his title is redundant: experienc- 
ing subjectively is the same as being conscious.) Like most 
philosophers, mathematicians, and physicists who expatiate on 
consciousness, he shows no sign of having considered what was 
shown, and how it was shown, by any psychological experiment on 
the perceiver’s achievement in a visual task. He also ignores the 
philosophical advances following the later Wittgenstein’s debunk- 
ing, 60 years ago, of the pervasive fallacy of supposing that when 
a patch that is red (in the world that we all live in) is seen as red, 
this is a “seeming” in another world (Lyons 1983). Worse, because 
these appearances, subjective experiences, conscious qualia, or 
whatever, are part of each of us, Lehar (like many) locates them in 
our heads, or as neurocomputations if we are foolish enough to 
look for consciousness among the brain cells (Booth 1978). This is 
all a big mistake about the grammar of the verb “to seem.” When 
we are viewing something but have reason to doubt that we per- 
ceive it correctly, then we may retreat to a claim that it seems to 
be so. We are not looking at a world inside our minds; we are hav- 
ing problems in seeing the colour of the patch out there. 

The grammar of “seeming as though” or “seeing as” also shows 
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what the subjective experience is isomorphic to. The syntax of “as” 
is the figure of speech known as simile. Subjective visual experi- 
ence is holistic, at least at times, because the world in which we 
operate is “holistic” in its optics; black holes are pretty uncommon 
in everyday life. Lehar actually says this in section 3, although he 
has hidden the point from himself by a tangle of the conceptual 
mistakes that Wittgenstein (1953) cut through. “The perceptual 
experience of a triangle cannot be reduced to just three phenom- 
enal values but is observed as a fully reified triangular structure 
that spans a specific portion of perceived space” (sect. 3, para. 2). 
Delete the reference to a contrary and all the redundancies and 
we get: “The perceptual experience of a triangle... is... as 
[sic]... triangular... .” 

Furthermore, a triangle is not a triangle in any world unless it 
“emerges” “whole,” “real,” and “invariant.” If a Gestalt is taken to 
be a subjective experience (rather than a perceptual perfor- 
mance), then it is consciousness simply of “seeing the world as it 
is. 

There is no space in this commentary to dissect out the multi- 
tudinous errors built on this fundamental misorientation. Suffice 
it to deal with the absurdity of the target article’s Figure 2. Lehar 
shows phenomenological slapdash if not downright dishonesty. 
You know and I know that he has never looked one way down a 
road at the very same moment as looking the other way. So it is 
rank self-deception to write (sect. 6.3, para. 1) that “the two sides 
of the road must in some sense be [subjectively] perceived as be- 
ing bowed” as in the diagram. His Bubble bursts. 


Double, double, toil and trouble — fire burn, 
and theory bubble! 


Birgitta Dresp 

Etude des Mécanismes Cognitifs et Dynamique du Langage, UMR 5596 
CNRS-Université Lumiére, Lyon 2, France. birgitta.dresp@univ-lyon2.fr 
dresp@convergence.u-strasbg.fr 


Abstract: Lehar’s Gestalt Bubble model introduces a computational ap- 
proach to holistic aspects of three-dimensional scene perception. The 
model as such has merit because it manages to translate certain Gestalt 
principles of perceptual organization into formal codes or algorithms. The 
mistake made in this target article is to present the model within the the- 
oretical framework of the question of consciousness. As a scientific ap- 
proach to the problem of consciousness, the Gestalt Bubble fails for sev- 
eral reasons. This commentary addresses three of these: (1) the 
terminology surrounding the concept of consciousness is not rigorously 
defined; (2) it is not made evident that three-dimensional scene percep- 
tion requires consciousness at all; and (3) it is not clearly explained by 
which mechanism(s) the “picture-in-the-head,” supposedly represented in 
the brain, would be made available to different levels of awareness or con- 
sciousness. 


In this target article we are told that “the most serious indictment 
of contemporary neurophysiological theories is that they offer no 
hint of an explanation for the subjective experience of visual con- 
sciousness” (sect. 1, para. 2). Lehar attacks “good old” Neuron 
Doctrine by stating that as a theoretical approach to visual per- 
ception, it has reached a dead end because he (Lehar) finds it 
“hard to imagine how . . . an assembly of independent processors 
[neurons] could account for the holistic emergent properties of 
perception identified by Gestalt theory” (sect. 1, para. 3). He then 
proposes his own doctrine, the Gestalt Bubble model. The Gestalt 
Bubble is presented as a computational approach to the percep- 
tual representation of three-dimensional visual space using a vol- 
umetric matrix of dynamic elements, each of which can exist in 
one of several states: transparent for the representation of void 
space, opaque coplanar for the representation of smooth surfaces, 
opaque orthogonal for the representation of corners, and opaque 
occlusion for the representation of surface edges. The supposed 
transformation of the physical world outside by a perceptual 


process taking place inside the brain is defined as the turning on 
of the appropriate pattern of elements in the volumetric matrix of 
the model in response to visual input. The Gestalt Bubble thereby 
replicates the three-dimensionality of visual objects as they are 
experienced in the subjective percept. The principal merit of this 
model resides in the fact that it translates some major Gestalt laws 
of visual perception such as emergence, reification, multistability, 
and invariance into computational codes. 

What the author fails to make clear in his target article is the 
supposed link between his Gestalt Bubble model and general the- 
ories of consciousness. All he does here is demonstrate that mod- 
ern computer technology produces algorithms that allow us to 
translate the laws of perceptual organization formulated in Gestalt 
theory into formal codes within the framework of a computational 
model. What the model has to do with consciousness, however, re- 
mains totally unclear. Neither the fact that we are able to con- 
sciously experience and describe three-dimensional shapes as en- 
tities and wholes, nor the fact that we can find laws or codes 
describing how these emerge perceptually, implies or proves that 
consciousness is necessary to see and move around in three-di- 
mensional space. In addition, although Lehar seems to imply that 
his Gestalt Bubble provides a ready model of what he refers to as 
visual consciousness, he fails to provide clear definitions of what 
we are supposed to understand by visual consciousness, phenom- 
enal awareness, subjective perceptual experience, or conscious- 
ness in general. In the title of the target article, he uses the term 
“subjective conscious experience.” Does this suggest that there 
should be an objective conscious experience as well? Moreover, 
the author readily assumes the existence of a “visual conscious- 
ness” as a particular form of consciousness. This assumption needs 
to be justified. How would a visual consciousness operate in com- 
parison to an auditory, tactile, or olfactory consciousness, for ex- 
ample? In fact, by using ambiguous terminology in his text (ter- 
minological danglers?), switching readily from one level of 
explanation to another, the author fails to convince his readers that 
he knows what he is talking about when he discusses the question 
of consciousness. 

Moreover, the fundamental difference between Lehar’s “pic- 
ture-in-the-head” model and the concept of isomorphism from 
classic Gestalt theory is not discussed in a satisfactory manner. Af- 
ter a lengthy introduction that confronts the reader with odds and 
ends of numerous general theories of mind and consciousness, the 
author all of a sudden pops up his own version of the Gestalt hy- 
pothesis of isomorphism by suggesting that we see the outside 
world as we do because that is and has to be the way the world is 
represented in the brain. This “picture-in-the-head” view goes far 
beyond the classic Gestalt concept of isomorphism because it as- 
sumes not only a functional but also a structural correspondence 
between the visual percept and its brain representation. It is in- 
troduced here as the only rightful answer to Koffka’s question 
“Why do we see things as we do?”; the original Gestalt viewpoints 
(e.g., Kohler 1961; Metzger 1936; von Ehrenfels 1890; among oth- 
ers) on isomorphism are not discussed. 

Interestingly, the author seems to have overlooked that his “pic- 
ture-in-the-head” hypothesis (structural isomorphism) stands or 
falls on the validity of the assumption that one of the key princi- 
ples formulated by Gestalt theory, that of the common fate of parts 
(Ganzbestimmtheit der Teile; Metzger 1936), reflects the result of 
a neurophysiological mechanism. In the early sixties, some psy- 
chophysicists questioned the neurophysiological validity of pre- 
cisely this principle of perceptual organization. Pritchard (1961) 
presented figures as stabilized images on the retina and showed 
that the constituent elements of these figures disappeared from 
phenomenal awareness one by one — not all at once, as the prin- 
ciple of common fate of parts would predict if it reflected the re- 
sult of a neurosensory mechanism (see also Pritchard et al. 1961). 
In any case, even if the “picture-in-the-head” view could be 
proven right, Lehar would still have to come up with an explana- 
tion of the mechanism(s) by which the picture in the head is made 
available to consciousness. Also, a rigorous distinction between 
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“awareness,” such as awareness of the emergent properties of a vi- 
sual object at a given moment, for example, and “consciousness,” 
such as the consciousness of being aware of the emergent prop- 
erties of a visual object and its significance within a general con- 
text, for example, would then have to be made. 

Lehar writes that it is of central importance for psychology to 
address what “all that neural wetware” is supposed to do and to 
determine which of the competing hypotheses presented in the 
introduction of his target article “reflects the truth.” Who said that 
science has to bother with metaphors such as “truth”? As far as I 
understand it, science is all about facts and measures collected 
within a specific context of boring constraints, usually called “con- 
ditions,” and therefore inevitably requires a diversity of methods 
and hypotheses. The concept of “truth” does not appear to be of 
much use here. Are we not often enough reminded to take care 
not to get trapped by the metaphors we use to construct hypothe- 
ses and explanations? The overwhelming “Unsumme” (as defined 
by Metzger 1936) of bits and pieces of philosophy and phenome- 
nological “brain teasers” we are confronted with in this target ar- 
ticle somehow shows how easily we can end up like the Sorcerer's 
Apprentice in Goethe's poem, who tries all sorts of curses and in- 
vokes all sorts of spirits, but is finally unable to take control. 

In conclusion, whether theories based on or derived from the 
Neuron Doctrine will ultimately fail to provide a satisfactory ap- 
proach to the question of consciousness, remains to be seen. The 
Gestalt Bubble model, as a scientific approach to consciousness, 
can be filed DOA (Dead on Arrival). 


NOTE 
1. After Shakespeare, Macbeth. 


Just bubbles? 
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Abstract: Lehar misrepresents the Neuron Doctrine and indirect realism. 
His conclusions on consciousness are unjustified. The Bubble Gestalt per- 
ceptual modeling disconnected from neuroscience has no explanatory 
power. 


1. Perception has not evolved for our enjoyment; it serves action, 
exploration of the world (see O'Regan & Née 2001). Although the 
richness of visual perception may partially be an illusion, sensory 
data should elicit brain states that reflect important features of 
perceptual organization. Such functional representation would be 
very useful, facilitating information retrieval from visual and au- 
ditory cortex, stored in attractor neural networks after termination 
of direct sensory inputs (Amit 1994). Persistent brain activity may 
be responsible for visual imagery, filling in, illusory contours, and 
other such phenomena. This internal representation, being a 
physical state of the brain, is focused and interpreted by other 
brain areas, gating it to the working memory and facilitating con- 
scious perception. It is constructed from sparse information ob- 
tained from eye fixations between saccades (as is evident in the 
change blindness experiments; O'Regan & Née 2001) and hence 
may not be as faithful and rich as it seems. Because for many peo- 
ple endowed with visual imagination (individual variance seems to 
be quite large in this respect) visual experiences are rich and vivid, 
filling in of missing information must be strong. 

2. Construction of the inner perspective is a difficult task. Lehar 
does not even attempt to enumerate the dimensions required for 
perceptual modeling that could replace (or at least complement) 
neural modeling. I have argued myself (Duch 1997) that an inter- 
mediate level of cognitive modeling should be useful. It should 
represent mental events in a way that is closer to our inner per- 
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spective, acceptable to psychologists, but should also facilitate re- 
duction, at least in principle, to the neural level. Complex neural 
systems reveal emergent processes (responsible, as Lehar has no- 
ticed, for Gestalt phenomena) requiring a higher level of descrip- 
tion characterized by new laws and phenomena. The usual ap- 
proximation to neural activity misses the perceptual level by going 
from states of recurrent networks (such as Grossberg’s adaptive 
resonant states; Grossberg 1995) to states of finite automata (cf. 
Parks et al. 1998 for neural models in psychiatry). A shortcut from 
neuroscience via neural networks to behavior is satisfactory only 
to behaviorists. Mind states and mental events may emerge as “a 
shadow of neurodynamics” in psychological or perceptual spaces 
(Duch 1997). This is in accord with the ideas of Shepard (1987; 
1994), who believes that universal laws of psychology may be 
found in appropriate spaces. Psychological spaces are spanned by 
subjective dimensions (such as color, shape, and motion), and one 
may use them to explain subjective perception and to talk about 
mental events implemented at the neurodynamical level. There- 
fore, I sympathize with Lehar’s goal, although details of his pro- 
posal are not satisfactory. 

3. Trivializing the “Neuron Doctrine,” Lehar writes about 
neural networks as the “quasi-independent processors,” and “an 
assembly of independent processors” (target article, sect. 1, para. 
3). The whole essence of neural networks is in the interaction of 
their elements, cooperative computational abilities that facilitate 
their holistic emergent properties. Recurrent neural networks are 
certainly not “the atomistic feed-forward model of neurocompu- 
tation” (target article, Abstract; cf. Parks et al. 1998). The Neuron 
Doctrine paradigm has been completely misinterpreted in the tar- 
get article. 

4. The arguments evoked against indirect realism are strange to 
say the least. Lehar mixes mental and physical levels freely, writ- 
ing statements like “the world that appears to be external to our 
head is actually inside our head” and “beyond those perceived sur- 
faces is the inner surface of your true physical skull encompassing 
all that you perceive” (sect. 2.2, para. 1). How can the physical 
skull encompass the nonphysical, inner world? “The world inside 
the head” is a metaphor, and it does not make much sense to in- 
vert it, unless one believes that there is some kind of physical 
world squeezed inside the skull. 

Indirect realism claims that we perceive and comment upon the 
states of our own brain. These states reflect properties of the en- 
vironment, but interpretation of the spatial structure of the states 
of the visual system has nothing to do with their physical location. 
There is nothing strange about it, as there is nothing strange about 
transmission of the voice and images via wires and radio waves. 
The spatial world inside the head is there in the same sense as a 
panoramic image in the integrated circuit of a computer graphic 
chip. Subjective reversal of a multistable percept follows the 
change of neural dynamics. It has to be experienced vividly as an 
inversion of a perceptual data structure, because visual experi- 
ences are a reflection of neural dynamics — how else could changes 
of visual cortex states be experienced? 

5. It is certainly not clear “that the most fundamental principles 
of neural computation and representation remain to be discov- 
ered” (target article, sect. 2.4, para. 3). Churchland (1984) had al- 
ready argued against it 20 years ago, and since that time compu- 
tational neuroscience has made a lot of progress. It may very well 
be that Hebbian learning is the only fundamental principle that is 
needed and that sufficiently complex models of the brain will be 
able to simulate its emergent functions. 

6. It is quite probable that “our own conscious qualia evolved 
from those of our animal ancestors” (sect. 6.5, para. 3). But cer- 
tainly the “conclusion” (sect. 6.5, para. 6) “that all matter and en- 
ergy have some kind of primal protoconsciousness” is not in- 
escapable. In fact, I regularly lose my consciousness in sleep, and 
anesthetics and damage to the reticular formation lead to coma, 
obliterating consciousness. Complex organization of matter is not 
sufficient for consciousness. Instead of looking for conditions nec- 
essary for manifestation of consciousness — a fruitful way is to use 
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here a contrastive approach between perception and reception 
(Taylor 1999) — Lehar goes down the beaten track of thinking 
about consciousness as some kind of a substance that is present in 
all matter, although sometimes in watered-down form. The con- 
clusion of this line of reasoning is absurd: protoconsciousness of 
soap bubbles. 

Of course, because the concept of consciousness is not defined, 
one may try to extend it to all matter, but talking about stomachs 
being “conscious” leaves no semantic overlap with the word “con- 
scious” applied to a baby, or to a cat. If consciousness is a function 
and plays a functional role, as Lehar seems to believe (“It seems 
that conscious experience has a direct functional role” — sect. 6.5, 
para. 10), the inescapable conclusion is rather that not all brains 
are equal. Language is unique to humans, and even though one 
can extend the concept of language to some more primitive forms 
of communication, interaction between internal organs of the 
body or messages passing between components of a computer sys- 
tem is not the same “language” as natural languages. The differ- 
ence between a “field” in agriculture and “field” in physics is com- 
parable to the difference between animal “consciousness” and 
“consciousness” of a soap bubble due to the physical forces that 
determine its shape. We should not be deceived by words. 

7. It remains to be seen if the main contribution of the target 
article, the Gestalt Bubble model, will be useful for understand- 
ing or even for a description of perception. The goal of science is 
not modeling per se but rather explaining and understanding phe- 
nomena. Modeling perception should not become an exercise in 
computer graphics, creating volumetric representations of space 
and objects. Bubbles of neural activity, as presented by Taylor 
(1999), have real explanatory power and are amenable to empiri- 
cal tests. The perceptual modeling proposed by Lehar promises a 
new language to describe high-level visual perception. Any lan- 
guage that is useful in design and analysis of experiments must re- 
flect more basic neural processes. Nothing of that sort has been 
demonstrated so far, and it is doubtful that the Gestalt Bubble 
model can explain observations that have not been hidden in its 
premises. 


Empirical constraints for perceptual 
modeling 


Charles R. Fox 

Whitely Psychology Laboratories, Franklin and Marshall College, Lancaster, 
PA 17604-3003. charlie.fox@fandm.edu http://www.fandm.edu/ 
departments/psych_new/faculty/fox/ 


Abstract: This new heuristic model of perceptual analysis raises interest- 
ing issues but in the end falls short. Its arguments are more in the Carte- 
sian than Gestalt tradition. Much of the argument is based on setting up 
theoretical straw men and ignores well known perceptual and brain sci- 
ence. Arguments are reviewed in light of known physiology and traditional 
Gestalt theory. 


Steven Lehar’s article purports to present a new model of per- 
ception based on Gestalt principles. Lehar raises some interesting 
issues but in the end falls short of his claims. His heuristic model 
is more Cartesian than Gestalt and much of his argument is based 
on setting up straw men. He ignores much of what is known in 
perceptual and brain science. I will confine myself to these issues, 
although there are others. 

Lehar maintains the Cartesian mind-body distinction and as- 
sumes internal representation as a requirement. He also ignores 
the distinction between conscious perception as active construc- 
tion and the perception/action continuums implied by physiology 
and direct perception data. Lehar recycles the Cartesian ma- 
chinelike body now inhabited by the “ghosts” of mental represen- 
tations and computations. This dualism is at odds with traditional 
Gestalt theory (Kéhler 1969). 


The target article ignores the contemporary distinction be- 
tween (1) perceptual mechanisms that subserve action; and (2) the 
cognitive mechanism of recall and analysis; instead, it suggests the 
latter as the sole perceptual mechanism. This emphasis stems 
from Lehar’s belief that “introspection is as valid a method of in- 
vestigation as is neurophysiology” (sect. 2.3, last para.). This is not 
the position of traditional Gestalt theory, which states that “a sat- 
isfactory functional interpretation of perception can be given only 
in terms of biological theory” and warns that “The value of bio- 
logical theories in psychology is not generally recognized.” Gestalt 
psychology adopted the program of building bridges between psy- 
chological rules and the activities of the central nervous system 
(Kohler 1940; 1947; 1961). Kéhler recognized this task as “beyond 
present technical possibilities.” These purely technical limits are 
being overcome today, yet the target article ignores a large body 
of empirical physiological evidence, some of which is presented 
below (see also Milner & Goodale 1995 and Gallese et. al. 1999 
for summary of some areas). Although we should not limit our the- 
ories to physiology, theory must account for known physiology. 
The target model does not. To take a specific example, the model 
ignores the important role of eye movements even though they 
were of concern to the early Gestalt theorists (Koffka 1935) and 
are a critical part of contemporary perceptual theory (Ebenholtz 
2001). More generally, there is ubiquitous evidence, collected 
over many decades, for the important role of physiological systems 
in perception. Simply consider the differential perceptions re- 
sulting from anatomical and physiological states of sensory end or- 
gans. Visual perception in the myopic, dark-adapted, or macular- 
degenerated eye is more influenced by anatomy and physiology 
than by computations on a mental image. 

Lehar emphasizes computational neuroscience at the expense 
of known physiology despite his assertion that “most fundamental 
principles of neural computation and representation remain to be 
discovered” (sect. 2.4, para. 3). This leads to oversimplification to 
the point of error. For example, he dismisses direct perception be- 
cause “No plausible mechanism has ever been identified neuro- 
physiologically which exhibits this incredible property” (sect. 2.2, 
para. 3) and “all that computational wetware” (sect. 2.1, para. 2) 
must serve some “purpose” (i.e., “produce an internal image of the 
world’; sect. 2.1). Yet there is growing physiological evidence to 
the contrary. As I have discussed elsewhere (Fox 1999), area MST 
in monkeys (similar to area V5 in humans) shows cells that are re- 
sponsive to three-dimensional motion information that is charac- 
teristic of the type of flow field emphasized by direct perception 
theory (Duffy & Wurtz 1995; 1997a; 1997b). More recently, direct 
perception theorists have examined the relation of neural infor- 
mation systems to Tau, a property of environmental optics (Gre- 
aly 2002; Lee et al. 2002). Hence, contemporary physiology sup- 
ports an emerging model suggestive of an environmentally 
adapted physiology rather than the metaphor of representational/ 
computational “wetware.” 

Lehar further misrepresents direct perception theory as de- 
scribing perception “as if perceptual processing occurs somehow 
out in the world itself rather than as a computation in the brain” 
(sect. 2.1, para. 1). Using the term “perceptual processing” or 
“computation” is a serious misrepresentation of direct perception 
(Gibson 1966; 1979), regardless of where one attributes it. Gibson 
contends that the perceptual system is sensitive to “affordances” 
that are naturally occurring and require no processing but rather 
are directly perceived. The exact characteristics of affordances are 
disputed, but a recent paper (Chemero 2003) provides a critical 
analysis and comprehensive definition of the concept of affor- 
dances and makes it very clear that affordances are perceived re- 
lations that are dynamic but neither computed nor components of 
computations. This is consistent with the physiology described 
above. 

Gestalt psychology is also misrepresented as a representational/ 
computational approach. I content that a key — perhaps the key — 
insight of Gestalt theory is that adequate knowledge of wholes, 
such as objects, comes from observing wholes. Such understand- 
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ing does not come from a “humpty-dumpty” approach that tries to 
put the object “back together again” through computation. The 
target model is reductionist/empiricist and, as such, is contrary to 
Gestalt theory (Koffka 1935; Kéhler 1947). The relevant proper- 
ties of things are not computational properties superimposed on 
the object system, but rather, the intrinsic relational properties 
within the object and between the object and the perceiver/actor 
(Kohler 1947). For example, Kéhler certainly did not suggest that 
perception is a mental computation when he wrote: “While climb- 
ing once in the Alps I beheld... a big dark cloud... nothing 
could be more sinister and more threatening. . . . the menace was 
certainly in the cloud.” The menace stems not from computations 
on mental images but from physiological sensitivity to relations 
among environmental physical energies, and between these rela- 
tions and the state system of the observer/actor. I suggest a dy- 
namic, person-environmental mechanism rather than internal 
representation and computations. This is consistent with the 
Gestalt statement: “rules in which we formulate (functional, psy- 
chological) relationships imply occurrences of certain functions in 
a realm that is surely not the phenomenal realm” (Kéhler 1940). 

A final, critical point concerns isomorphism: Isomorphic rela- 
tions are ubiquitous, so one needs to be specific. Gestalt “Psy- 
chophysical Isomorphism” is a hypothesis that rejects Cartesian 
dualism and is informed by physiology (Kéhler 1969). Lehar, us- 
ing a digital computer metaphor, suggests a point-to-point iso- 
morphism between the internal image and external objects/space. 
However, this is not supported by physiology. Cells in the supple- 
mentary eye field of the monkey show firing patterns (Olson & 
Gettner 1995) that do not encode visual space in any one-to-one 
manner. Rather, they incorporate higher dimensions of informa- 
tion such as attention or purpose (Fox 1999). Hence, even if we 
accept isomorphic, internal representations, there is neurophysi- 
ologic evidence that such representations are more complex than 
suggested in Lehar’s model. 

The target model does not accomplish its ambitious goals of 
presenting a modern Gestalt perceptual model. A more fruitful 
heuristic for understanding perception is a physiology that has 
evolved a sensitivity to meaningful environmental relational in- 
formation, or, as suggested by Clark (1998), one that represents 
action-oriented systems. 
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Linking visual cortex to visual perception: 
An alternative to the Gestalt Bubble 


Stephen Grossberg 
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Abstract: Lehar’s lively discussion builds on a critique of neural models of 
vision that is incorrect in its general and specific claims. He espouses a 
Gestalt perceptual approach rather than one consistent with the “objec- 
tive neurophysiological state of the visual system” (target article, Abstract). 
Contemporary vision models realize his perceptual goals and also quanti- 
tatively explain neurophysiological and anatomical data. 


Lehar describes a “serious crisis,” “an impasse,” and a “theoretical 
dead end” (target article, sect. 1, para. 1) in contemporary models 
of vision and advances as a possible alternative his Gestalt Bubble 
approach, “which is unlike any algorithm devised by man” (Ab- 
stract). He also claims that “Gestalt aspects of perception have 
been largely ignored” (sect. 1, para. 3) by neural models of vision, 
and then goes on to describe presumed dichotomies between 
equally desperate attempts to understand how the brain sees. 
Lehar particularly comments about modeling work by my col- 
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leagues and myself, noting that “the most serious limitation of 
Grossberg’s approach . . . is that, curiously, Grossberg and his col- 
leagues did not extend their logic to . . . three-dimensional spatial 
perception [and] ... no longer advocated explicit spatial filling- 
in” (sect. 3, para. 5). He also says it is “impossible for Grossberg’s 
model to represent transparency” (sect. 3, para. 5). These general 
and specific claims unfortunately do not accurately represent the 
published literature about neural vision models. Lehar seems mo- 
tivated to trash neural vision models because his own model makes 
no contact with neurophysiological and anatomical data about vi- 
sion. 

In reality, there is an emerging neural theory of three-dimen- 
sional vision and figure-ground perception called the FACADE 
theory, for the multiplexed Form-And-Color-And-DEpth repre- 
sentations that the theory attempts to explain (Grossberg 1987; 
1994; 1997). Lehar refers to my 1994 article in summarizing the 
deficiencies of our models. However, this article explains many 
three-dimensional figure-ground, grouping, and filling-in per- 
cepts, including transparency, and uses an explicit surface filling- 
in process. Later work from our group has developed these qual- 
itative proposals into quantitative simulations of many 
three-dimensional percepts, including three-dimensional per- 
cepts of da Vinci stereopsis, figure-ground separation, texture seg- 
regation, brightness perception, and transparency (Grossberg & 
Kelly 1999; Grossberg & McLoughlin 1997; Grossberg & Pessoa 
1998, Kelly & Grossberg 2000; McLoughlin & Grossberg 1998). 

These studies laid the foundation for a breakthrough in under- 
standing how some of these processes are organized within iden- 
tified laminar circuits of cortical areas V1 and V2, notably 
processes of cortical development, learning, attention, and group- 
ing, including Gestalt grouping properties (Grossberg 1999a; 
Grossberg & Raizada 2000; Grossberg & Seitz 2003; Grossberg & 
Williamson 2001; Grossberg et al. 1997; Raizada & Grossberg 
2001; 2003; Ross et al. 2000). 

This LAMINART model has been joined with the FACADE 
model to develop a three-dimensional LAMINART model that 
quantitatively simulates many perceptual data about stereopsis 
and three-dimensional planar surface perception, and functionally 
explains anatomical and neurophysiological cell properties in cor- 
tical layers 1, 2/3A, 3B, 4, 5, and 6 of areas V1 and V2 (Grossberg 
& Howe 2003; Howe & Grossberg 2001), using three-dimensional 
figure-ground and filling-in concepts to do so. More recently, the 
three-dimensional LAMINART model has been generalized to 
explain how three-dimensional percepts of slanted and curved 
surfaces and of two-dimensional images are formed, and to clar- 
ify how three-dimensional grouping and filling-in can occur over 
multiple depths (Grossberg & Swaminathan 2003; Swaminathan 
& Grossberg 2001). This work includes explanations of how iden- 
tified cortical cells in cortical areas V1 and V2 develop to enable 
these representations to form, how three-dimensional Necker 
cube representations rival bi-stably through time, how slant after- 
effects occur, and how three-dimensional neon color spreading of 
curved surfaces occurs even at depths that contain no explicit bot- 
tom-up inputs. All these studies are consistent with the grouping 
interpolation properties that Kellman et al. (1996) have reported 
(p. 51), and with the three-dimensional grouping properties sum- 
marized in Lehar’s Figure 16, which he seems to think cannot yet 
be neurally explained. 

These modeling articles show that many of the perceptual goals 
of Lehar’s Gestalt Bubble model are well handled by neural mod- 
els that also provide a detailed account of how the visual cortex 
generates these perceptual effects. In summary, we do not need 
analogies like the soap bubble (sect. 8.2), or rod-and-rail (sect. 8 
and Fig. 6), or different local states to represent opaque or trans- 
parent surface properties, as Lehar proposes. The brain has dis- 
covered a much more interesting solution to these problems, 
which links its ability to develop and learn from the world with its 
ability to see it. 

Lehar makes many other claims that are not supportable by pre- 
sent theoretical knowledge. He claims that “we cannot imagine 
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how contemporary concepts of neurocomputation ... can ac- 
count for the properties of perception as observed in visual con- 
sciousness [including] hallucinations” (sect. 2.4, para. 3). Actually, 
current neural models offer an explicit account of schizophrenic 
hallucinations (Grossberg 2000) as manifestations of a breakdown 
in the normal processes of learning, expectation, attention, and 
consciousness (Grossberg 1999b). 

Contrary to Lehar’s claims in section 8.7, recent neural models 
clarify how the brain learns spatial representations of azimuth, el- 
evation, and vergence (see Lehar, Fig. 14) for purposes of, say, 
eye and arm movement control (Greve et al. 1993; Guenther et al. 
1994). Lehar defends “the adaptive value of a neural representa- 
tion of the external world that could break free of the tissue of the 
sensory or cortical surface” (sect. 8.8). Instead, What stream rep- 
resentations of visual percepts should be distinguished from 
Where stream representations of spatial location, a distinction 
made manifest by various clinical patients. 

Lehar reduces neural models of vision to capacities of comput- 
ers to include navigation as another area where models cannot 
penetrate (see sect. 6.1 and sect. 9). Actually, neural models quan- 
titatively simulate the recorded dynamics of MST cortical cells 
and the psychophysical reports of navigating humans (Grossberg 
et al. 1999), contradicting Lehar’s claim that “the picture of visual 
processing revealed by the phenomenological approach is radi- 
cally different from the picture revealed by neurophysiological 
studies” (sect. 9, para. 1). In fact, a few known properties of corti- 
cal neurons, when interacting together, can generate emergent 
properties of human navigation. 

Lehar ends by saying that “curiously, these most obvious prop- 
erties of perception have been systematically ignored by neural 
modelers” (sect. 10, penultimate para.). Curiously, Lehar has not 
kept up with the modeling literature that he incorrectly charac- 
terizes and criticizes. 


Steven Lehar’s Gestalt Bubble model of 
visual experience: The embodied percipient, 
emergent holism, and the ultimate question 
of consciousness 


Keith Gunderson 


Department of Philosophy, University of Minnesota, College of Liberal Arts, 
Minneapolis, MN 55455-0310. gunde002@umn.edu 


Abstract: Aspects of an example of simulated shared subjectivity can be 
used both to support Steven Lehar’s remarks on embodied percipients and 
to triangulate in a novel way the so-called “hard problem” of consciousness 
which Lehar wishes to “sidestep,” but which, given his other contentions 
regarding emergent holism, raises questions about whether he has been 
able or willing to do so. 


Steven Lehar’s Gestalt Bubble model (GBM) is said to emphasize 
the often ignored fact “that our percept of the world includes a 
percept of our own body within that world, ... and it remains at 
the center of perceived space even as we move about in the ex- 
ternal world” (sect. 6.4). I offer here a friendly, if folksy, example 
of a simulation of shared first-person subjectivity designed to re- 
inforce Lehar’s brief but interesting claims concerning the promi- 
nence of the embodied percipient in visual perception. This ex- 
ample leads to other questions regarding his analysis. I have 
labeled the example elsewhere, and with variations, the Cinematic 
Solution to the Other Minds Problem, and invoked it earlier 
against B. F. Skinner's view of subjective privacy and scientific in- 
quiry, also objected to by Lehar for his own reasons (Gunderson 
1971; 1984). 

Suppose a film director wishes to treat us to the subjective per- 
ceptual experiences of another person, say Batman, as he gazes on 
the traffic far below from some window perch. How is this best 
done? Not, to be sure, by simply showing us the whole scene: the 


superhero perched on the ledge with the traffic moving by on the 
street below. This would not be anything like being privy to Bat- 
man’s subjective perceptual experience. It would only amount to 
our own visual experience coming to include Batman. Instead, 
what is characteristically done is that Batman’s filmed body (or at 
least the better part of it) is somehow (gradually or suddenly) sub- 
tracted from the screen in such a manner that we become insinu- 
ated into roughly whatever space and orientation Batman’s body 
occupies and are thereby made party to the visual field (sense of 
height, traffic passing below, etc.), which we can assume would be 
Batman’s from that perspective. We cannot, of course, literally oc- 
cupy (even cinematically) exactly the same space that Batman 
does — a prerequisite to having his visual experience — but the 
tricks of the art permit us to enjoy a simulation of such an occu- 
pancy. It is the sleight-of-camera with respect to our seemingly 
ubiquitous embodied presence in visual perception that carries 
with it tactics for conjuring a sense of the usual “subjectivity bar- 
rier” between us and another percipient being breached. And 
here it occurs in a florid phenomenological manner, obviously dif- 
ferent from the “relational information” that can cross that barrier, 
as described by Lehar (sect. 5.1). Notice too, that a “preset” fea- 
ture of the whole typical movie experience involves the darkened 
theater and no focused sense of our own body being either pre- 
sent in the audience or included in the screen action. The effect 
is that where we are not assuming specifically Batman’s perspec- 
tive, we are assuming one belonging to no one in particular, or 
rather one “belonging” to anyone in the vicinity, as it were. 

So the possibility of the cinematic simulation of shared subjec- 
tivity seems to presuppose the inclusion of an embodied percipi- 
ent in our visual perceptions, along lines suggested by Lehar. But 
the apparent friendliness of the example has a complicated 
provocative side as well. For if what it takes to create the illusion 
is the clever collapsing of our perspective (or someone else’s) into 
another's, then the epistemic-ontic primacy of the first-person 
point of view becomes obvious, and the “hard problem” of con- 
sciousness can be rephrased with respect to it this way: There is 
no analogous thought experiment that would render subjectivity 
or a point of view (one’s own or another's) as being somehow man- 
ifest in any set of neurophysiological processes to begin with, such 
that another consciousness might appear as somehow insinuated 
into it. But there should be, if consciousness is to be modeled (dis- 
played, illustrated) within any third-person physicalistic concep- 
tualization. This rather flat and crude-sounding point is not, I 
think, irrelevant or naively realistic. In a nutshell, that there can 
be no cinematic-type simulation of a solution to the mind-body 
problem parallel to another mind’s, can be seen to stem from our 
inability to cling to our sense of experiencing a point of view while 
being in some neurophysiological locus (however this is repre- 
sented). 

For Lehar, the salient residual problem(s) is this: Although the 
contents of all our subjective visual experience for the GBM are 
subsumed under the subjective, we lack any vivid demonstration 
of how having a first-person point of view in itself, which is a pre- 
requisite to there being any such phenomenal contents, lies within 
that experience. Simply specifying underlying neurophysiological 
conditions for consciousness takes us nowhere we have not already 
unsatisfactorily been. That there is, and how there is, any locus at 
all for our perceptions remains unexplained within any micro or 
macro frame of reference. We think, of course, that the locus of 
our locus of perceptions lies in some way within the embodied. 
But to be apprised of all this does not thereby help us to see how 
any subjective perspective occurs in the first place, or why it is 
uniquely ours! (See Nagel 1965.) The problem of explaining it 
arises independently of whatever type of metaphysical substance 
the perceiver is believed to be embodied in, even as part of a 
panpsychic or panexperientialist scheme such as Chalmers’ (as in 
sect. 6.5). And it can be reiterated with respect to any type of sub- 
stance of any kind of complexity, as far we can tell. 

Now, Lehar wishes to “sidestep” these latter matters by casting 
the GBM wholly within the subjective. Our perceived worlds — 
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our pattern-recognizing activities, including, of course, our total 
physical natures — will then supposedly lie within the range of 
what his subjectively rendered model is a model of. But I do not 
see how this really matters, even when naive realism such as Skin- 
ner’s is deleted from the picture for the (laudable) reasons Lehar 
provides. One might, of course, wish, out of other considerations, 
simply to set the mind-body problem aside and concentrate on re- 
fining taxonomic characterizations within phenomenal experi- 
ence. (Nagel 1974 is cited as having suggested something like 
this.) 

But more puzzling to me is why Lehar’s concluding remarks 
about Koffka’s and KGhler’s views on emergence (sect. 7.1), which 
Lehar finds more satisfying than Davidson’s anomalous monism, 
are not a way of directly addressing “the hard problem.” The piv- 
otal demystifying image in the “bottom-up” aspect of Lehar’s sum- 
mary of the mind-body relationship is that of perception charac- 
terized along Gestalt lines as being related to neurophysiological 
processes, in the way that a soap bubble holistically emerges from 
“a multitude of tiny forces acting together simultaneously” to pro- 
duce a final perceptual state by way of a process that cannot be re- 
duced to simple laws (sect. 7.1). But whatever other, if any, pur- 
poses this no doubt interesting image may serve, the relationship 
between bubble and tiny forces is not in any discernible way sim- 
ilar to whatever the connection between subjective states of con- 
scious perceptual awareness and neurophysiological states is like. 
Both bubbles and tiny forces are happily in the world, as it were, 
whether as macro-bubblistic ones, or micro-force-istic ones, or as 
something like the pop-out dog example (sect. 7.1). These all in- 
volve one set of “out there” aspects being related to other “out 
there” aspects, whether within the subjectivized purview of the 
GBM or some other one. The bugbear of consciousness still seems 
to turn on the point that first-person conscious perspectival states 
cannot yet be even imagined as either macro or micro anythings 
to begin with, much less as popping up from micro ones. 


Backdrop, flat, and prop: The stage for active 
perceptual inquiry 


Julian Hochberg 


Columbia University, 170 West End Avenue, New York, NY 10023. 
Hochberg@columbia.edu 


Abstract: Lehar’s revival of phenomenology and his all-encompassing 
Gestalt Bubble model are ambitious and stimulating. I offer an illustrated 
caution about phenomenology, a more fractured alternative to his Bubble 
model, and two lines of phenomena that may disqualify his isomorphism. 
I think a perceptual-inquiry model can contend. 


Steven Lehar’s ambitious Bubble metaphor is highly stimulating, 
assuming a unified phenomenal visual world that explains and pre- 
dicts our perceptual experience. Herewith are a cautionary re- 
minder about phenomenology as such, an alternative to Lehar’s 
specific enclosing Bubble model, and two lines of phenomena that 
Lehar ignores but that are difficult to reconcile with the particu- 
lar isomorphism he espouses. 

Phenomenology should indeed guide psychophysics and neu- 
rophysiology. But phenomenology is certainly not incontestable. 
For example, Lehar cites the CIE chromaticity diagram as a de- 
scription of phenomenological color space. The Helmholtzean 
dogma — that the experience of yellow consists of red plus green 
experiences — lurked within mainstream sensory physiology until 
after World War II (and was often attributed to the CIE). Then, 
following Hering instead, Hurvich and Jameson’s (1957) phe- 
nomenologically guided opponency-oriented psychophysics and 
model explained to neurophysiologists what their microelectrodes 
later revealed, thereby changing our view of neurophysiology and 
liberating our relevant phenomenology. (In fact, Jameson & Hur- 
vich showed later [1967] that the CIE is no phenomenological 
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Figure 1 (Hochberg). Onstage and backdrop scenery. A. The 
strength of the major depth cues with egocentric distance, 
adapted from Cutting and Vishton 1995 (with permission). To the 
eye as actor, the backdrop usually lies between 10 and 50 feet up- 
stage. B. The experienced stage in which visual inquiry proceeds. 
The viewer's normal actions provide no distance information be- 
yond the plane labeled “backdrop” and they can readily generate 
and therefore incorporate information about the downstage prop. 
The curves of Figure 1A account for but are not salient in the ex- 
perience of B. C. Attention extends the stage. When the inquir- 
ing eye visits a scene, its boundaries are remembered as further 
out than they were (see Intraub 1997); this is not merely memory, 
because such Boundary Extension (BE) is a function of where the 
viewer plans to look (Intraub et al. 2001). 


summary — two very different colors come out at the same point 
on the graph.) Phenomenology must be both consulted and con- 
tested. Accordingly, a different model follows. 

Lehar’s tackling of encompassing space is an important step, but 
other phenomenological details might support a different, less 
holistic model — a stage or set, not a bubble: Several quite differ- 
ent aspects of our visual ecology afford distance information. 
Their zones of efficacy, as in Figure 1A (after Cutting & Vishton 
1995), are surely important for any account of our encompassing 
visual world. Assume that the furthest zones form an essentially 
equidistant region like the backdrop on the stage in Figure 1B. 
Railroad tracks visible in those zones appear to converge. In 
nearer zones, the depth information effectively specifies the 
tracks as parallel and holds the backdrop in its place upstage. 

This implies discontinuities (e.g., between backdrop and stage) 
that are not firmly fixed, because where the viewer attends, and 
with what intentions, affects what information is recovered and 
used (cf. Fig. 2B, C). Figure 1A can therefore serve only as a con- 
ditional account; and as Figure 1C implies, the phenomenal lay- 
out itself varies somewhat with the viewer's perceptual intentions. 
In this model, therefore, distance to the end of the internal world 
is not a continuous variable nor continuously defined. Why aren’t 
the discontinuities spontaneously evident? 

Is there evidence of such overlooked discontinuity? Figure 2A 
seems to reverse as a whole and has been offered as one example 
of how a minimum principle (including Lehar’s version) leads to 
perceiving an entire three-dimensional structure (Hochberg & 
MacAlister 1953; Kopfermann 1930). But Figure 2B shows that, 
when tested, perfectly possible objects display the same depen- 
dence on what the viewer attends as was previously shown by the 
Penrose and Penrose (1958) impossible figures. Perceptual conse- 
quences (Hochberg 1998; in press), such as the effects of rotation 
described in Figure 2B and the surface-lightness effect in Figure 
2C, attest that these are perceptual phenomena. They also share 
some aspects of Lehar’s isomorphism. (And the absence of any 
salient break between the different spatial zones of the environ- 
ment in Figures 1A and 1B, and in the apparently-continuous 
bubble that Lehar describes, merely parallels what happens 
within objects.) 

Such phenomena raise difficulties for any holistic proposed iso- 
morphism powered by the physical relationships as perceived. 
Gestaltist visions of isomorphism were of course concerned 
mostly with flat shapes, not three-dimensional structures (see 
Hochberg 1998). The fact that Peterson and her colleagues (see 
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Figure 2 (Hochberg). Some shapes isomorphism must take. A. 
The reversible Necker cube. Sometimes offered as an example of 
how a minimum principle (or something like it, in Lehar’s version) 
leads to perceiving an entire three-dimensional structure (Hoch- 
berg & McAlister 1953; Kopfermann 1930). B. The partly re- 
versible Killer cube. When attended at (a), the present cube ap- 
pears of definite and nonreversible three-dimensional structure; 
when attended at (b), it soon starts reversing, though the same 
Gestalt remains in view (though off attentional center). The re- 
versals are attested by their perceptual consequences: When ro- 
tated clockwise around its vertical axis, the perceived motion is 
clockwise when (a) is attended; when (b) is attended and when it 
appears nearest the viewer, motion appears counterclockwise. 
Such perceptual consequences help validate one’s otherwise un- 
supported phenomenology, as in the next figure (Hochberg, in 
press; Hochberg & Peterson 1987). C. Adelson’s Impossible Stair- 
case. With no discernible discontinuity, the right and left sides 
here are incompatible as three-dimensional structures; showing 
that they are actually seen that way. Note that the same print den- 
sity appears of higher reflectance (lighter paint job) at (b) than at 
(a) — (after Adelson 2000, with permission); see text. D. Do con- 
figuration-based organizational factors first provide figure-ground 
segregation, which thereby offers a shape to be recognized? Not 
so you can tell: see text (see Peterson 1994; Peterson & Gibson 
1993). 


Figure 2D; cf. Peterson 1994; Peterson & Gibson 1993) have 
shown that meaningful (denotative) shapes preempt figural status 
when in their familiar orientations (Fig. 2Dc,d) but not when the 
physically identical configurations are inverted (Fig. 2Da,b), 
makes it hard even to imagine what an appropriate formulation of 
isomorphism would be like. A phenomenology centered on query- 
directed units of perceptual behavior, emulating the TOTE rubric 
offered by Miller et al. (1960), might be more effective (cf. 
Hochberg 1970; in press; O’Regan & Née 2001). 


Does perception replicate the external world? 


Donald D. Hoffman 

Department of Cognitive Science, University of California at Irvine, Irvine, CA 
92697. ddhoff@uci.edu 
http://aris.ss.uci.edu/cogsci/personnel/hoffman/hoffman.html 


Abstract: Vision scientists standardly assume that the goal of vision is to 
recover properties of the external world. Lehar’s “miniature, virtual-real- 
ity replica of the external world inside our head” (target article, sect. 10) 
is an example of this assumption. I propose instead, on evolutionary 
grounds, that the goal of vision is simply to provide a useful user interface 
to the external world. 


Lehar asserts that “The central message of Gestalt theory is that 
the primary function of perceptual processing is the generation of 
a miniature, virtual-reality replica of the external world inside our 
head, and that the world we see around us is not the real external 
world but is exactly that miniature internal replica” (target article, 
sect. 10, last para.). I wish to consider this assertion of indirect re- 
alism. 

Suppose it is true. Then we do not see the real external world, 
nor do we hear, smell, taste, or in any other way perceive it. In- 
stead, we perceive just the miniature virtual-reality (henceforth, 
mini VR) that we generate. Given this, what empirical grounds 
might we have for claiming that our mini VR replicates the exter- 
nal world? Perhaps we could compare objective measures of the 
external world against psychophysical measures of the mini VR. If 
mismatches are minor, we would have grounds for the replica 
claim. This process seems straightforward enough. The basic sci- 
ences measure the external world, and psychology the mini VR. 
So we simply compare data. 

But this is too fast. It is not just psychologists who perceive only 
their mini VRs; all scientists, regardless of discipline, perceive only 
their mini VRs. So how do the basic scientists manage to measure 
the external world? 

The trouble is that every time scientists try to measure the ex- 
ternal world, whether they look through telescopes or micro- 
scopes, they see only their mini VRs. They extend their senses with 
countless technologies, but the technologies and their outputs are 
still confined to the mini VRs; for if they were not, then, accord- 
ing to indirect realism, the scientists could not perceive them. 
Hence, all scientists are confined to perceive only their mini VRs. 
If they wish to make assertions about the external world, even as- 
sertions that an external world exists, then these are necessarily, 
according to indirect realism, theoretical assertions. They are not 
direct measures. As Einstein notes, “physics treats directly only of 
sense experiences and of the ‘understanding’ of their connection. 
But even the concept of the ‘real external world’ of everyday think- 
ing rests exclusively on sense impressions” (Einstein 1950, p. 17). 

So indirect realism does not allow us incontrovertible empirical 
grounds to assert that our mini VRs replicate the external world. 
At best, it allows us to postulate an external world as a theoretical 
construct. Once we take the external world as a theoretical con- 
struct, then we have many options for the particular form of that 
construct. We can, as Lehar suggests, propose that our mini VRs 
are replicas of the external world. This is a particularly simple the- 
ory and, on the face of it, quite unlikely. Our best evidence sug- 
gests that mini VRs vary dramatically across species (Cronly- 
Dillon & Gregory 1991), and there are no evolutionary grounds to 
suppose that our species happens to be the lucky one that got it 
right. To assert otherwise would be anthropocentric recidivism. 

Once we extend our gaze beyond the replica theory, many other 
possibilities arise. One class of possibilities is that there is little or 
no resemblance whatsoever between the external world and our 
mini VRs, but that instead our mini VRs are simply useful user in- 
terfaces to the external world, with no more need to resemble that 
world than a Windows interface needs to resemble the diodes, re- 
sistors, and software of a computer. Of course, we could not call a 
theory from this class an “indirect realist” theory because, by hy- 
pothesis, there is no realism. So indirect realism leads us to con- 
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sider dropping indirect realism in favor of a broader and more 
likely class of theories. Let us call these new theories “user-inter- 
face” theories. For what they entail is that our mini VRs, rather 
than being replicas of the external world, are simply useful user 
interfaces to that world. Different species employ different user 
interfaces for their different purposes. The human user interfaces 
are simply a small set of the total, of special interest to us for only 
parochial reasons. 

The move from indirect realism to user interface can be dis- 
concerting, for it denies an anthropocentrism very dear to us: the 
assumption that our perceptions are privileged among all species. 
And it opens a Pandora’s box of theoretical possibilities for the na- 
ture of the external world and its relation to our mini VRs. It has 
been convenient to assume that because there are neurons and 
synapses inside the heads that appear in our mini VRs, therefore 
there must be corresponding real neurons in real heads in the ex- 
ternal world. But convenience rarely coincides with truth. It 
looked for millennia as though the sun and stars circled the earth, 
but we now know better. Even space and time themselves are not 
immune from this process, for as Einstein pointed out: “Time and 
space are modes by which we think and not conditions in which 
we live” (quoted in Forsee 1963, p. 81). 

Moving from indirect realism to user interface does nothing to 
impede progress in modeling of the mini VR itself along the 
Gestalt lines proposed by Lehar. Nor does it impede progress in 
modeling the neural networks of the perceptual systems in our 
mini VRs. All this modeling can continue as it has. We simply re- 
alize that we are not modeling a replica of the external world; we 
are instead modeling our species-specific user interface to an ex- 
ternal world. And in consequence we are far more cautious in our 
knowledge claims about the external world. 

The move from indirect realism to user interface gives us more 
elbowroom in dealing with the hard problem of consciousness. 
The hard problem arises when we assume that neurons as we per- 
ceive them in our mini VRs are replicas of real neurons in the ex- 
ternal world, and we must therefore figure out how those real neu- 
rons could possibly give rise to conscious experience. But if we 
drop the replica assumption, we now have a broader range of the- 
oretical possibilities for what, in the external world, might corre- 
spond to neurons in our mini VRs. In this case our only limits in 
solving the problem are not the straitjacket of the replica as- 
sumption, but our imaginations. 
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Psychological relativity 


Donald Laming 


Department of Experimental Psychology, University of Cambridge, 
Cambridge, CB2 3EB, United Kingdom. drji@cus.cam.ac.uk 


Abstract: “Psychological relativity” means that “an observation is a rela- 
tionship between the observer and the event observed.” It implies a pro- 
found distinction between “the internal first-person as opposed to the ex- 
ternal third-person perspective.” That distinction, followed through, turns 
Lehar’s discourse inside-out. This commentary elaborates the notion of 
“psychological relativity,” shows that whereas there is already a natural sci- 
ence of perceptual report, there cannot also be a science of perception per 
se, and draws out some implications for our understanding of phenome- 
nal consciousness. 


Lehar is lacking an essential idea. Physicists have it — “relativity” 
— but Lehar does not. Lehar mentions (sect. 1) “the internal first- 
person as opposed to the external third-person perspective” but 
fails to realise how that distinction impacts on his discourse. If the 
implications of that distinction are followed through, the entire 
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Figure 1 (Laming). The different views from four houses on a 
housing estate. (© 2004, Donald Laming. Reproduced with per- 
mission from D. Laming, Understanding human motivation, 


Blackwell.) 


body of problems addressed is turned inside-out. The overriding 
principle that Lehar is lacking is: 
an observation is a relationship between the observer and the event ob- 
served 


and thereby depends on the observer as well as the event. So, two 
observers in motion relative to each other make different deter- 
minations of the velocity of a third object (Galilean relativity). Fig- 
ure 2 sketches the set-up for Thouless’s (1931a; 1931b) phenom- 
enal regression to real size. The observer has a different view of 
the experiment to the experimenter. 

Figure 1 presents an analogy. Looking out from my window, I 
can see three other houses, separated from me by a road and a 
green sward. If there is a car in the road, my neighbour and I can 
readily agree that it is red. By agreeing on a suitable instrument 
for measurement, we can agree the colour of the car to whatever 
precision we desire. That arena outside our houses (camera view) 
is part of the public domain within which experiments can be con- 
ducted. But my neighbour and I cannot see into each other's 
houses. If I telephone my neighbour, I can only describe my inte- 
rior furnishings by reference to what my neighbour will have seen 
elsewhere. The scope of experimental procedure can be extended 
to internal experience only by projecting that experience into the 
public domain. I might describe my curtains as scarlet, or carmine, 
or cerise — but my neighbour might think of a different colour ref- 
erent to the one that I have in mind, and “seeing red” will then 
mean slightly different things to the two of us. 

I can invite my neighbour into my house to see for himself but 
I cannot give him direct access to my visual experience. One might 
suppose that my internal visual experience could be measured, 
like the colour of the car in the road. But experimental psycholo- 
gists have been trying to measure internal sensations for 150 years 
and have so far progressed nowhere (Laming 1997). 

Some part of our visual experiences can be shared with others; 
the remainder is private. The Gestalt properties surveyed in sec- 
tions 5 and 7 belong to that private part, which is why Gestalt psy- 
chology has not proceeded beyond verbal description. There is a 
boundary between experiences that can be shared and experi- 
ences that are essentially private. It is determined by what, within 
my field of view, my neighbour can also see (see Fig. 1). That is, 
the boundary is determined within my neighbour's field of view 
and is not to be found within my own visual experience. My own 
experience by itself contains no distinction between that which 
lies in camera view and that which is private. The junction is seam- 
less. It is only too easy to confound subjective experience with ob- 
jective observation; this is what Lehar has done. 
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It follows that there cannot be a natural science of perception. 
There is a science of perceptual report, a tradition that goes back 
to Fechner (1860/1966). But perceptual reports cannot be taken 
at their face value (here the Gestalt psychologists erred); rather, 
they must be evaluated by experiment. Lehar is aware of this (sect. 
5.2), but asserts that perceptual experience is isomorphic to the 
neural substrate and thereby denies this distinction. 

Lehar’s stance is that “the world of conscious experience is ac- 
cessible to scientific scrutiny after all, both internally through in- 
trospection and externally through neurophysiological recording” 
(sect. 2.3, para. 9). He envisages an isomorphism between per- 
ceptual experience as described by the observer and the observa- 
tions of the natural scientist. Thouless’s (1931la; 1931b) experi- 
ment on phenomenal regression to real size (Fig. 2) shows why 
such an isomorphism is not found in nature. 

The observer's task is to select a disc set normal to the line of 
sight at distance a to match the angular size of the larger disc at 
distance b. Although people do choose a smaller disc from the al- 
ternatives at a, they systematically choose one too large to match 
(phenomenal regression to real size). Imagine that a neurophysi- 
ologist is making observations at the neural level of description rel- 
evant to understanding how and why this error of judgment oc- 
curs. If the observer's perceptions stand in the same relation to the 
neural substrate as the neurophysiological observations, then 
there has to be an internal “observer” looking at internal processes 
with the same objectivity as the neurophysiologist. The fact that 
Lehar has a mathematical model to replace the neurophysiologi- 
cal observations does not alter this requirement. This observer is 
represented by the “thinks bubble” in Figure 2. Philosophers will 
immediately identify this internal observer as Ryle’s (1949) “ghost 
in the machine” (which is why the “thinks bubble” is decisively 
crossed out). 

I next ask whether the hypothetical neurophysiologist can also 
observe the neural substrate of this “ghost.” If so, the relationship 
of the ghost to the neural substrate is structurally different from 
that of the neurophysiologist; otherwise the “ghost” is pure mind- 
stuff. In fact, verbal descriptions of what is perceived are pro- 
duced by the same system as that which does the perceiving, and 
the relationship of “observer” (if that term may still be used) to the 
neural substrate that is supposedly “observed” is essentially differ- 
ent from that of a third-party neurophysiologist. Several conclu- 
sions follow: 

There need not be any useful isomorphism between neural 
process and perceptual experience. 

Modelling perceptual experience is not an alternative to un- 
derstanding the neural process. 

There cannot be a natural science of perception, distinct from 
the study of perceptual report. 

The idea of psychological relativity also impacts on conscious- 
ness (sect. 6). Because it is impossible to access any other person's 





Figure 2 (Laming). 
of phenomenal regression to real size. (© 2004, Donald Laming. 
Adapted with permission from D. Laming, Understanding human 
motivation, Blackwell.) 


Experimental set-up for the measurement 


subjective experience, it is not possible to observe any other per- 
son’s consciousness. Even if the hypothetical neurophysiologist 
were to observe and record a substrate in the brain that subserved 
consciousness, there is no way in which the observations could be 
identified as such. However much one explores the brain, all that 
one finds is brain function. Phenomenal consciousness is simply 
the quality of subjective experience. 

Lehar’s discourse has neglected some real empirical relations 
between perceptual report and experimental observation. I give 
two examples. Rubin (1921) drew attention to the “figure-ground” 
phenomenon, the assertion that the first stage in visual perception 
was the separation of a figure from its background. Elementary 
neurophysiological study has revealed that sensory neurons are 
differentially coupled to the physical input (Laming 1986), so that 
they are specifically sensitive to boundaries in the visual field 
while responding with only a noise discharge to uniform illumina- 
tion. This appears to match the “figure-ground” phenomenon. 
Second, the Necker cube is ambiguous as a visual stimulus. The 
ambiguity is temporarily resolved by factors from within the per- 
ceiver (sect. 7.3). But there is no reason why those internal factors 
should be consistent, comparing one instance with another, so that 
the project of constructing a consistent geometry of subjective 
perceptual space is not achievable. 


Double trouble for Gestalt Bubbles 


Dan Lloyd 

Department of Philosophy, Trinity College, Hartford, CT 06101; Helsinki 
Collegium for Advanced Studies, University of Helsinki, Finland. 
dan.lloyd@trincoll.edu 


Abstract: The “Gestalt Bubble” model of Lehar is not supported by the 
evidence offered. The author invalidly concludes that spatial properties in 
experience entail an explicit volumetric spatial representation in the brain. 
The article also exaggerates the extent to which phenomenology reveals a 
completely three-dimensional scene in perception. 


The real world is a place of many properties; so also is its presen- 
tation as a phenomenal world in the conscious brain. One way for 
a brain state to present in experience a worldly property P is to du- 
plicate P itself. Like a painter striving for perfect mimesis, an em- 
bodied consciousness might use patches of red in the head to rep- 
resent a red apple. Or, according to Lehar, a brain might use 
spatial properties to represent external spatial reality: 
The central message of Gestalt theory is that the primary function of 
perceptual processing is the generation of a miniature, virtual-reality 
replica of the external world inside our head, and that the world we see 
around us is not the real external world but is exactly that miniature in- 
ternal replica. (target article, sect. 10) 


Lehar’s article makes the case for the internal replica, or “Gestalt 
Bubble,” and then develops a model of how three-dimensional 
spatial modeling could occur in something like a neural medium. 
In this commentary, I suggest that the evidence in support of the 
Gestalt Bubble is in double trouble. It is both conceptually and 
phenomenologically flawed. 

The coffee in the cup at my elbow is (to me) hot, brown, of a 
certain weight and size, and in a specific location. We cannot con- 
clude, however, that the state of my brain that is my consciousness 
of the coffee replicates any of these properties itself. Yet this is an 
inference Lehar seems to make repeatedly in the target article. 
For example: “The fact that the world around us appears as a vol- 
umetric spatial structure is direct and concrete evidence for a spa- 
tial representation in the brain” (sect. 5.2). 

This is a non sequitur, as can be seen by substituting “colored” 
for “spatial” in the passage. A slightly more elaborate argument is 
no less fallacious: 
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The volumetric structure of visual consciousness and perceptual in- 
variance to rotation, translation, and scale offer direct and concrete ev- 
idence for an explicit volumetric spatial representation in the brain, 
which is at least functionally isomorphic with the corresponding spatial 
experience. (sect. 5.1) 


Lehar is right that functional isomorphism between phenomenal 
experience and its implementation is required to avoid “nomo- 
logical danglers,” but once again, “explicit volumetric spatial rep- 
resentation” is in no way entailed — for “rotation, translation, and 
scale” substitute “hue, saturation, and brightness,” and the fallacy 
will be apparent. Nor does Lehar’s claim that phenomenal spa- 
tiality preserves the relational structure of spatial objects entail an 
internal replica, because (once again) a three-dimensional rela- 
tional structure defines “color space” without in the least imply- 
ing that the color solid appears somewhere in our brain. Func- 
tional isomorphism, meanwhile, is readily preserved between 
spatial objects/scenes and their representations without invoking 
replicas. For example, the World Wide Web is well stocked with 
virtual worlds that preserve functional isomorphism with spatial 
scenes, each of them encoded in some nonspatial computational 
idiom such as VRML. 

In sum, the conceptual arguments in the target article do not 
support the author's main conclusion. Nonetheless, the brain does 
have properties, and some of its properties do determine the con- 
tents of conscious experience. Lehar’s arguments do not establish 
that the brain must use space to represent space. Does phenom- 
enality license any inferences at all about the neural medium? 
There are two ways to approach this question, beginning either 
with contingent generalities about perception or with its essential 
structures. The first approach begins with features of phenome- 
nality (as revealed by perceptual psychology, including the Gestalt 
demonstrations of our perceptual capacities). The second analysis 
isolates essential or necessary structures of phenomenality. The 
second approach accords with classical phenomenology, as exem- 
plified in the works of Husserl (e.g., Husserl 1974). In either case, 
the hope is that the analysis of phenomena will constrain the 
search for computational architectures sufficient to generate 
some or all of the features of phenomenality. 

On neither approach is there compelling reason to posit the spa- 
tial virtual world proposed by Lehar. I do not doubt that I live in 
a spatial world, but my visual field — that is, what I see before me 
right now — conveys far less spatial information than Lehar’s 
Gestalt Bubble encodes. At the focus of attention I am aware of 
surfaces, distance from my eyes, and edges, but outside of focal 
attention I experience only a very indefinite spatiality, which 
seems to me to be inconsistent with the continuously present 
three-dimensional models constructed in the Gestalt Bubble. The 
supposition that my experience specifies a full 360-degree dio- 
rama in my head arises from the “just-in-time” availability of spa- 
tial information with every attentional focus. The information is 
there when and where I need it, and experience presents an or- 
dered sequence of focally attended presentations rather than a 
single wraparound replica of the spatial world. This seems to be 
phenomenologically “given” but it is also amply confirmed in psy- 
chological studies of “inattentional blindness” (Mack & Rock 
1998) and “change blindness” (Simons 2000). (Sect. 8.8 briefly ac- 
knowledges the effect of successive gaze fixations in different di- 
rections, suggesting that parts of the replica fade while outside the 
visual field. This suggests either that the replica has an absolute 
spatial orientation and does not turn with the head or, if the replica 
does turn with the eyes, that only a small focal part of it has the 
spatial detail Lehar describes.) 

This disagreement can be made more rigorous and more prop- 
erly phenomenological. One essential property of the phenome- 
nal world is expressed in our ability to distinguish properties by lo- 
cation. That is, I can be aware of a red circle and a green square 
at the same time without confusing the pairings of colors and 
shapes. Austen Clark refers to the problem posed by this perva- 
sive perceptual ability as the “Many Properties” problem, and he 
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argues that it can be solved only by coding places along with other 
perceptual properties (Clark 2000). So “red” and “circle” must be 
assigned a location, and “green” and “square,” a second location. 
Experience, of course, solves the Many Properties problem easily, 
and arguably it is essential to the very concept of phenomenality 
that consciousness solve it. This argument so far provides support 
for Lehar’s position but immediately raises the question: How 
many spatial dimensions are required? Lehar advocates three, 
Clark suggests two, but the argument necessitates just one, a lin- 
ear dimension along which one point is tagged “red” and “circu- 
lar,” and another “green” and “square.” The basic dimension, then, 
would be temporal, and experience would be an orderly ensem- 
ble of phenomenal leaps and bounds, a time line. Spatiality 
emerges from trajectories encoded in proprioception that orient 
each momentary percept to those before and after. This proposal 
conforms well with classical phenomenology (Husserl 1966; 1974), 
and in other work, I present evidence for its implementation in 
the brain (Lloyd 2002; 2003). This alternative cannot be defended 
here, but it does suggest that the Gestalt Bubble is not entailed by 
phenomenology. 

It is important that theories of perception accommodate the 
Gestalt observations; Lehar brings forward an essential array of 
examples to consider, and exhibits the care and detail required to 
translate spatial perception into a computational model. But more 
evidence to support the model — from philosophy, phenomenol- 
ogy, psychology, and neuroscience — will be needed. 


Isomorphism and representationalism 


Riccardo Luccio 


Dipartimento di Psicologia, Universita’ degli studi di Firenze. 50123 Florence, 
Italy. luccio@cesit1 .unifi.it 


Abstract: Lehar tries to build a computational theory that succeeds in of- 
fering the same computational model for both phenomenal experience 
and visual processing. However, the vision that Lehar has about isomor- 
phism in Gestalttheorie as representational, is not adequate. The main 
limit of Lehar’s model derives from this misunderstanding of the relation 
between phenomenal and physiological levels. 


Gestalt psychology has been fundamentally misunderstood in the 
United States (though the field too has to bear some responsibil- 
ity; see Kanizsa 1995). After World War IL, it had a meager des- 
tiny, cultivated only marginally in Germany and America though 
more intensively in peripheral countries such as Italy and Japan. 
However, mainly in the last few decades, some concepts of Gestalt 
psychology have appeared frequently in psychological debate, 
such as priignanz, isomorphism, minimum principle, and so forth. 
The continuing debate demonstrates the inability of cognitive psy- 
chology to accept some highly significant aspects of our way of 
picking up the reality that is around us. Lehar’s paper does not 
confine itself to stressing the importance of some classic Gestaltist 
ideas taken in isolation, as other scholars in the past have done, in 
an attempt, never completely successful, to integrate part of the 
Gestalttheorie into cognitive psychology. Instead, Lehar tries to 
build a computational theory that succeeds in offering the same 
computational model to both phenomenal experience and visual 
processing. 

This highly interesting attempt deserves some comment, how- 
ever. In my opinion, Lehar’s vision of Gestalttheorie is not fully ad- 
equate, and this has some consequences for his theorizing. The 
point on which I disagree almost completely with Lehar is the fol- 
lowing: He claims that there is a central philosophical issue that 
underlies discussions of phenomenal experience, as seen, for ex- 
ample, in the distinction between the Gestaltist and the Gibson- 
ian view of perception. Is the world we see around us the real 
world itself or merely an internal perceptual copy of that world 
generated by neural processes in our brain? In other words, this 
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is the question of direct realism, also known as naive realism, as 
opposed to indirect realism, or representationalism. I note paren- 
thetically that although Gibson (1966; 1979) called himself a naive 
realist, this was only a provocation. The theory of direct percep- 
tion is neither naive nor realistic. As Michaels and Carello (1981, 
p- 90) clearly put it, “the test of the veridicality of perception con- 
cerns the mutual compatibility of the action of the actor/perceiver 
with the affordances of the situation.” Here we are very far from 
the veridicality requested by genuine naive realism. 

More important is the picture of Gestalt psychology that Lehar 
offers to us. It is well known that in Gestalttheorie there was a 
strong Spinozian attitude. For example, Wertheimer (greatly im- 
pressed by Spinoza’s Ethica from childhood on: see Luchins & 
Luchins 1982) remained in this orientation all his life. So we can 
speak in terms of an indifference or “indifferentism” about the 
problem of representation. In general, Gestaltist isomorphism has 
to be considered as a variant of psychophysical parallelism (see 
Boring 1942; 1950, mainly Ch. 13; for a recent survey of this issue, 
see Luchins & Luchins 1999). But the same could be said about 
almost all other Gestalt psychologists. Lehar quotes Kéhler ex- 
tensively. But Kéhler never said that “the world we see around 
us... (is)... generated by neural processes in our brain” (target 
article, sect. 2, para. 1). Kéhler, indeed, was in some instances a 
little ambiguous on this topic (e.g., Kéhler 1969). But he was ab- 
solutely clear when he had to address the mind-body problem di- 
rectly. He conceived the Gestalt position as a variant of parallelism 
(KGhler 1960, pp. 20-21), and said: “The thesis of isomorphism as 
introduced by the Gestalt psychologists modifies the parallelists’ 
view by saying that the structural characteristics of brain processes 
and of related phenomenal events are likely to be the same” (em- 
phasis added). 

Lehar, quoting Kohler (1969), insists that the isomorphism re- 
quired by Gestalt theory is not a strict structural isomorphism but 
merely a functional isomorphism. But Kohler always spoke of 
structural isomorphism. He was very clear in stating (Kéhler 1940, 
Chs. 2 and 3) that the processes that run in our brain do not have 
any necessary correlate in our phenomenal experience. What is 
structurally identical is their interaction with what happens in bor- 
dering areas of the brain and the interaction that there is in the 
phenomenal field: Their dynamics and the dynamics of the phe- 
nomenal field. 

The structural identity between the phenomenal world and 
physiological processes does not imply any causal relationship be- 
tween the two levels. It means only that we are made up of one, 
and only one matter. The physical laws that rule matter lead to 
structurally identical outcomes when we consider the phenome- 
nal level as well as the physiological one. In this sense, Gestalt psy- 
chology is neither representationalist nor antirepresentationalist; 
it is indeed indifferentist. 

The main limit of Lehar’s model derives, in my opinion, from 
this misunderstanding. His computational model, as I can assess 
it, works perfectly for a world that is organized in terms of soap 
bubbles (Koffka’s metaphor [Koffka 1935], used too by Attneave 
1982). A soap bubbles world is, in Gestalt terms, a world in which 
the forces of the perceptual field tend to dispose themselves to 
make an outcome that is maximally good, pregnant in the sense of 
ausgezeichnet. In Lehar’s model, this happens at the phenomeno- 
logical as well as the neurophysiological level. The fact is that — as 
I believe Kanizsa and I have demonstrated (Kanizsa & Luccio 
1986; 1990) — a tendency of this kind does not exist in perception. 
These tendencies are instead well present in thinking, in memory, 
in all that Kanizsa (1979, Ch. 1) called “secondary processes,” to 
distinguish them from primary processes of perception. But they 
are beyond the scope for which the concept of isomorphism is in- 
teresting — and relevant. 

In recent years, a few other computational models have been 
presented to account for some typically Gestaltist phenomena, 
from information theory, to coding theory, to group algebra. How- 
ever, Lehar is right when he says that they cannot account for both 
the phenomenal level and the neuropsychological level. I should 


stress that there is at least one exception: nonlinear dynamic sys- 
tems, and, in particular, the synergetic approach. Apparently, we 
have not yet at our disposal a fully comprehensive theory; it should 
be interesting to test if the model proposed by Lehar could be in- 
tegrated with other approaches. 


The unified electrical field 


William A. MacKay 


Department of Physiology, University of Toronto, Toronto, ON M5S 1A8, 
Canada. william.mackay@utoronto.ca 


Abstract: The electrophysiological perspective presents an electrical field 
that is continuous throughout the body, with an intense focus of dynami- 
cally structured patterns at the cephalic end. That there is indeed an iso- 
morphic mapping between the detailed holistic patterns in this field and 
in perception (at some level) seems certain. Temporal binding, however, 
may be a greater challenge than spatial binding. 


The independent processor model of individual neurons has given 
rise to the widespread impression, echoed by Lehar, that neuro- 
physiology fails to deliver a unified basis for the holistic properties 
of perception. If there is any “illusion,” it is not in the unity of per- 
ceptual awareness but in the portrayal of physical separation by 
techniques such as extracellular recording and fMRI. Overlooked 
is the axis of continuous activity stretching from the spinal cord to 
the cerebrum. The tonic activity in the brain stem activating sys- 
tems (cholinergic, serotonergic, and noradrenergic), plus the his- 
taminergic activating system of the hypothalamus, is responsible 
for our state of (un)consciousness (Pace-Schott & Hobson 2002). 
All sensory and motor activity feeds into this axis and influences 
the general distribution of activity. Also, the activating systems can 
directly trigger synchronization of activity within the cerebral cor- 
tex (Munk et al. 1996). 

Furthermore, it is extremely doubtful that action potentials are 
of much significance in the direct link to perception. They are far 
too fleeting. It is the more sustained membrane potentials that are 
likely to correlate the best. Discrete neuronal activity in the brain, 
however isolated it may appear, is simply a local distortion in an 
unbroken continuum of electrical flux. All cells produce mem- 
brane potentials, even if static, such that an electrical field en- 
compasses the entire body. The “panexperientialism” view would 
also suggest that perceptual awareness is linked to something like 
an electrical field. This is the only obvious property that is shared 
by both the atom and the organism, and it is increasingly elabo- 
rated as one ascends to the organism. One might postulate that the 
higher the degree of complexity in the electrical field, the higher 
the level of consciousness experienced. Using fMRI, it can be seen 
that the same cortical areas are active whether a stimulus is per- 
ceived or not. The difference in the case of perception is that the 
level of activation is greater (Moutoussis & Zeki 2002). This could 
mean either that more neurons are depolarized within the given 
area, or that the same synapses are active, but at a higher fre- 
quency, or both. 

Neurons and their attendant glial cells manipulate membrane 
potentials like no other part of the body. This is their “game.” 
Many attributes of neuronal electrical activity extend the range of 
information coding. No single one of them is the essence of con- 
scious perception, but collectively they can raise (or lower) the 
level of consciousness. Spike synchrony is unquestionably rele- 
vant. For example, Riehle et al. (2000) have shown that unit pairs 
in the motor cortex synchronize activity to a very significant de- 
gree exactly at the moment of an expected signal. However, syn- 
chrony is not essential for “binding.” In area MT, Thiele and 
Stoner (2003) recorded from pairs of units, one pair preferring the 
direction of motion of one visual grating, and the other preferring 
another grating direction. The units did not usually synchronize 
activity when the gratings were perceived as moving together in a 
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coherent plaid. Synchrony elicited by coherent plaids was the 
same as for noncoherent ones. Again, it is probably not spiking ac- 
tivity per se that is ultimately important, but the associated 
changes in membrane potential and possibly phenomena such as 
depolarization fields manifested in superficial layers of cortex 
(Roland 2002). 

The various states of Lehar’s Gestalt Bubble model can easily 
be construed as hypothetical neuronal feature detectors. One 
could not ask for a better set of discriminators of planar proper- 
ties in depth, and I suspect that something very similar lurks some- 
where in the association areas between V1 and the inferotempo- 
ral cortex. The transformation from a two-dimensional image on 
the retina to a three-dimensional percept would follow a process 
as outlined by Lehar when the stimulus is an everyday, familiar ex- 
perience with established expectations. For any unfamiliar object, 
whether presented to the eye or hand, exploratory movement is 
requisite to clarify ambiguities. Here, Lehar is correct to empha- 
size the translation/rotation invariance of the perception, divorced 
from the motion of the explorer. The target is perceived as it re- 
lates to its environment external to the viewer. This is the essence 
of the great transformation from egocentric (parietal cortex) to al- 
locentric representation (presumably in the hippocampus or pre- 
frontal cortex). The constancy of the percept over time as another 
data sample is added with each exploratory movement is also 
rightly highlighted. 

It is essential that perception integrate over time as well as 
space. Even within one sampling episode, different sensory attri- 
butes such as color and motion are processed at slightly different 
times, although they are perceived as a unity. Hence, Zeki and 
Bartels (1998) postulate the existence of multiple “microcon- 
sciousnesses” in the brain, which are asynchronous with one an- 
other. This raises the problem of how they are integrated. A sim- 
ple possibility is that everything processed within a finite window 
is integrated, just as two colors flashed within less than 40 mil- 
liseconds are blended together. But it cannot be that simple, be- 
cause haptic exploration of an object can continue for hundreds of 
milliseconds. 

Figure-ground designation also involves time constraints. Neu- 
rons in the inferotemporal cortex that are selective for shape main- 
tain that shape preference when light-dark contrast is reversed 
(negative image) but not when a figure-ground reversal is made. 
Just as the perception of shape depends on whether a visual re- 
gion is assigned to an object or background, so the visual analysis 
of form depends on whether a region is perceived as figure or 
ground (Rubin 2001). One cannot relegate the problem of resolv- 
ing border-ownership of edges to earlier stages in the visual 
stream. It occurs quickly, within 10-25 milliseconds of response 
onset and really requires feedback from higher cortical areas. 
Hence, it is an instantaneous, holistic decision of the entire visual 
system, presumably selecting the most probable choice. 

Lehar’s excellent model of perceptual processes gives neuro- 
physiology some precise goals and direction. Hopefully, the out- 
come will be convincing evidence that every percept is associated 
with a unique distribution of neuronal activity. An immediate 
problem, however, is the elucidation of the mechanism for bind- 
ing elements of a percept in time. 


The soap bubble: Phenomenal state or 
perceptual system dynamics? 


Slobodan Markovié 


Laboratory of Experimental Psychology, Faculty of Philosophy, University of 
Belgrade, Belgrade, 11000, Serbia and Montenegro. smarkovi@f.bg.ac.yu 


Abstract: The Gestalt Bubble model describes a subjective phenomenal 
experience (what is seen) without taking into account the extraphenome- 
nal constraints of perceptual experience (why it is seen as it is). If it intends 
to be an explanatory model, then it has to include either stimulus or neural 
constraints, or both. 
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While presenting the theoretical background of his approach, 
Lehar attempts to keep a critical equidistance toward both indi- 
rect and direct realism. However, instead of a radically new ap- 
proach, he offers a combination of some constructivist and some 
Gibsonian premises. On the one hand, like many constructivists 
(e.g., Gregory 1971; Hochberg 1978; Marr 1982; Rock 1983), 
Lehar adopts a representational paradigm that defines perception 
asa subjective conscious description or as an internal virtual copy 
of the external world. On the other hand, inconsistent with the 
constructivists’ perspective and more similar to the views of pro- 
ponents of direct realism (e.g., Gibson 1979; Shaw & Bransford 
1977; Shaw & Turvey 1981), Lehar does not postulate any medi- 
ating mechanisms that process the representations within a per- 
ceptual system. 

Moreover, Lehar’s exact position concerning the question of di- 
rect perception of distal objects is not quite clear. At one point he 
explicitly claims that “the internal perceptual representation en- 
codes properties of distal objects rather than of a proximal stimu- 
lus” (sect. 9, last para.). At another point he states that “the direct 
realist view is incredible because it suggests that we can have the 
experience of objects out in the world directly, beyond the sensory 
surface, as if bypassing the chain of sensory processing” (sect. 2.1, 
para. 1). Why would the thesis that distal objects are mapping onto 
the phenomenological domain without neural intervention be in- 
credible and mysterious, while the idea about the projection of in- 
ternal representation onto the external perceptual world not be in- 
credible and mysterious? How is it possible that perception is 
partially indirect (representational), and partially direct (distally 
oriented)? 

In his criticism of neurophysiologic modeling, Lehar rejects not 
only the classical Neuron Doctrine, but also some recent holistic 
approaches (cf. Crick & Koch 1990; Eckhorn et al. 1988; Singer 
1999). Hence, for Lehar, the greatest shortcoming of neural mod- 
els is not atomism, but rather, the problem of neurophenomenal 
decoding. That is, how can a fully spatial (topographical) percep- 
tual description be created from spatially less constrained (topo- 
logical), or even completely abstract, symbolic, and nonspatial 
neural representation? I find that this epistemological question is 
a natural consequence of a hidden ontological dualism: How does 
one domain of reality (consciousness) know how to read and un- 
derstand the codes coming from the other (neural) domain? 

To paraphrase Koffka (1935), the ultimate task for perceptual 
science is to answer why things look as they do. In the case of 
Lehar’s theory, this question might be formulated as the following: 
Why is the phenomenal volumetric space such as it is? Why is it 
nonlinear in a particular way? Implicitly, Lehar proposes that this 
is an intrinsic property of phenomenal space which is not in a 
causal relationship with any other domain of reality. My opinion is 
that without the precise specification of the extraphenomenolog- 
ical aspects of perception, such as the stimulus and neural do- 
mains, it is difficult to answer the question related to why the per- 
cept looks as it does. For instance, imagine the difficulty in 
explaining the path shape and velocity of the planet Earth’s mo- 
tion without taking into account the mass and motion of other cos- 
mic objects (moon, sun, other planets, and so on). A description 
of the Earth’s motion is not an explanation of its motion. 

Even Gestalt psychologists, who widely utilized the phenome- 
nological method, did not create pure phenomenological expla- 
nations of perception. For instance, Koffka (1935) used the soap 
bubble metaphor, not to describe some phenomenal bubblelike 
experience, but to point out some basic principles of perceptual 
(neural) system functioning. Attneave (1982) also used the 
metaphor “soap bubble system” to describe the economy of per- 
ceptual system behavior. Like the soap bubble, which tries to en- 
close the largest volume within the smallest surface, the percep- 
tual system tends to reduce the global spending of energy 
(entropy, minimum tendency) while at the same time striving to 
increase its effective use (dynamics, maximum tendency) (cf. K6h- 
ler 1920/1938; 1927/1971; see also Hatfield & Epstein 1985; 
Markovié & Gvozdenovié 2001). 
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If Lehar intends to create a Gestalt-oriented theory of percep- 
tion, he has to have in mind that according to the classics of Gestalt 
theory, the phenomenological Gestalten are the consequences of 
both internal (neural) and external (stimulus) constraints (Koffka 
1935; Kéhler 1920/1938; 1927/1971; 1947). Simply speaking, the 
perceptual system tends to attain the maximum efficiency with the 
minimum investment (internal neural economy), but the minima 
and maxima will always be relative to the given stimulus conditions 
(external stimulus organization). The effect of external “control” 
of a perceptual economy is an articulation of more or less prég- 
nant Gestalten, or as Wertheimer stated in his famous Law of 
Priignanz, the phenomenal organization of a percept will be as 
“good” as the prevailing conditions allow (cf. Koffka 1935). 


Bursting the bubble: Do we need true 
Gestalt isomorphism? 


Niall P. McLoughlin 
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Abstract: Lehar proposes an interesting theory of visual perception based 
on an explicit three-dimensional representation of the world existing in the 
observer's head. However, if we apply Occam’s razor to this proposal, it is 
possible to contemplate far simpler representations of the world. Such 
representations have the advantage that they agree with findings in mod- 
ern neuroscience. 


Lehar proposes to model visual perception using his subjective vi- 
sual experience as his source of data. He proposes a perceptual 
modeling approach because “conventional concepts of neural pro- 
cessing offer no explanation for the holistic global aspects of per- 
ception identified by Gestalt theory” (target article, Abstract). 
This allows him conveniently to ignore current research in visual 
neuroscience while concentrating on the central issues of the rep- 
resentation of the visual field and of our subjective visual experi- 
ences. As he correctly points out, the world we see and experience 
surrounding us exists only as nerve impulses within our head. 
Lehar proposes that because our subjective experience of the 
world is that of a high-resolution three-dimensional volume, and 
because this representation must exist in our heads, it must there- 
fore be some form of a high-resolution three-dimensional struc- 
ture. However, this does not necessarily follow. For example, on a 
computer system it is possible to generate a sparse representation 
of the world into which it is placed so that the computer could in- 
teract with objects in the world in a meaningful manner. Objects 
could be represented as tokens at such-and-such x, y, and z loca- 
tion, and so forth. There would be no explicit representation of 
empty space within this sparse representation. Who is to say what 
the subjective experience of the computer might be? 

There is no doubt that my subjective experience of the world is 
that of a three-dimensional solid environment which I perceive in 
equal detail in all directions. Yet, as visual scientists and practiced 
observers, we know that this is patently not the case. Each of our 
eyes responds to incoming photons in a non-uniform manner and 
this non-uniformity is further exaggerated in the cortex. The over- 
representation of the fovea is magnified between the retina and 
cortex, and the multiple interconnected cortical regions amplify 
this distinction even further. Most naive observers are surprised 
to discover that they have a fovea and amazed that they have a 
blind spot in each eye. How do we fool ourselves? 

The very fact that we are genuinely fooled (until we make care- 
ful observations) calls into question the use of subjective experi- 
ence as the basis for theories of visual perception. Furthermore, 
although the Neuron Doctrine is indeed the foundation for most 
modern neuroscience research, I refute the notion that this doc- 
trine implies purely feed-forward models of neurocomputation. 


Certainly, recent findings in both neuroanatomy (e.g., Angelucci 
et al. 2002; Bosking et al. 1997) and neurophysiology (Kapadia et 
al. 2000; Levitt & Lund 1997) emphasize the roles played by feed- 
back and lateral connections in visual processing. Likewise, a 
number of popular modern computational theories make use of 
feed-forward, feedback, and lateral connections (e.g., Grossberg 
1994). Ifa Gestalt Bubble model subserves perception, then why 
do we have so many visual areas, each containing a retinotopic 
map of visual space? 

Is there any evidence for Gestalt-like processes at work neuro- 
physiologically? Recent electrophysiological recordings from as 
early as the lateral geniculate and V1 have found interactions well 
outside the classical receptive field (e.g., Blakemore & Tobin 
1972; Felisberti & Derrington 2001; Jones et al. 2000; 2001; Ka- 
padia et al. 2000; Levitt & Lund 1997; Solomon et al. 2002; Stet- 
tler et al. 2002). Although the source of these interactions 
(whether they are mediated by feedback or by lateral connections) 
remains to be elucidated, it is clear that many aspects of grouping, 
completion, and emergence may well arise from such nonlocal in- 
teractions. In addition, recent neurophysiological studies in the 
primate (e.g., Livingstone & Hubel 1988) suggest that different 
aspects of a visual scene are represented primarily in different vi- 
sual streams and areas. Although there is some disagreement as to 
the amount of segregation of function, numerous neuropsycho- 
logical studies in humans back up the suggestion that multiple rep- 
resentations exist for different attributes and/or functional roles. 
One such patient studied by Humphrey and Goodale (1998) suf- 
fered from visual-form agnosia (Farah 1990). She was unable to 
discriminate between visual forms, let alone recognize her friends 
and family, yet her color vision was close to normal and she could 
recognize shapes when placed in her hands. Such case studies sug- 
gest that the brain encodes the external world using multiple rep- 
resentations, each one perhaps subserving a different role or task 
rather than a single isomorphistic one. 

What Lehar seems to have forgotten is that the high-resolution 
representation is generated only when we pay attention to the in- 
put and focus our eyes on the object or texture under inspection. 
We need not represent even our immediate environment in high 
resolution unless we need to interact directly with it. Why waste 
time and space representing the world in vivid detail when we in- 
teract with only a small part of it at any one time? Surely our cen- 
tral representations should be goal-directed. We can always direct 
our vision to different locations in a scene to find out what is there, 
and given that most useful scenes are dynamic, why waste effort 
representing space in high resolution when it is constantly chang- 
ing? O’Regan (1992) argued along a similar line when he sug- 
gested that “seeing constitutes an active process of probing the en- 
vironment as though it were a continuously available external 
memory” (p. 484, emphasis in original). He suggests that seeing 
does not involve the reification of a three-dimensional spatial rep- 
resentation of the external world in the observer's head but rather 
depends on one’s ability to interrogate the environment through 
directed eye movements. It may well be that we have a fuzzy 
three-dimensional representation of the external world in our 
heads that we use to help direct eye movements, but I remain to 
be convinced that we would need or want anything more complex. 
If we need the detail, we look. 

Given the lack of physiological evidence for such a complex and 
computationally expensive representation, coupled with the lack 
of necessity for such a complete representation, Occam’s razor 
suggests we burst this Gestalt Bubble model. 
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Relations between three-dimensional, 
volumetric experiences, and neural 
processes: Limitations of materialism 
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Abstract: Certain features of perception — the quale red, for example, and 
other qualia — must be regarded as additions to the materialist neuro- 
physiological picture of perception. The perception of three-dimensional 
volumetric objects can also be seen as qualitative additions to the neuro- 
physiological processes in the brain, possibly without additions to the in- 
formation content. 


In the history of science and philosophy, the world has been re- 
garded as material, mental (idealist philosophy), or dualist (both 
material and mental). Like many people today, Lehar has chosen 
the materialist view, and he attempts to avoid dualism by assum- 
ing the mind-brain identity position (“consciousness is a physical 
process taking place in the physical brain” — sect. 2.3, para. 5). 
Still, he writes that there remains a subjective quality (or quale) to 
the experience of red, for example, which is not in any way iden- 
tical to any physical variable in the brain. I think this must mean 
that the experience of qualia adds something to the assumed ma- 
terial world and that Lehar therefore does not stay consistently 
within the materialist frame of reference. Lehar also writes (sect. 
2.3) that sense data, or the raw material of conscious experience, 
are the only thing we can know actually exists, and that all else, in- 
cluding the entire physical world, is informed conjecture based on 
that experience. To me this statement appears as a departure from 
materialism; it is actually close to the idealist view. 

I now suggest that the perceptual experience of three-dimen- 
sional, volumetric objects, and of empty space is also something 
that “subjective conscious experience” adds to the assumed mate- 
rial electrochemical processes in the brain, possibly without 
changing the information content — a qualitatively different rep- 
resentation. Lehar thinks that the gap between the materialist de- 
scriptions of neurophysiology and the phenomenological descrip- 
tions of Gestalt features of perception may be due to the present 
“embryonic” state of neurophysiology, but I regard this as a 
promissory belief rather than an explanation. 

Analogously (and staying within the materialist frame of refer- 
ence) I believe that a computer can produce a three-dimensional, 
volumetric figure, namely, if it is connected with a device that can 
construct that figure. The figure will then be another representa- 
tion of the information content which is represented inside the 
computer by electrical processes. Of course, a human person can 
also construct a three-dimensional figure with his hands or de- 
scribe it in words and drawings, as Lehar does. In this case, it is 
the connection with the body, particularly with the muscles and 
the hands, that enables the brain to make these constructions and 
descriptions from its information content. 

I think that materialism has served science well within a rather 
large domain, but with studies of cognition such as Lehar’s, we 
move into a domain where materialism reveals significant short- 
comings. I find that such shortcomings appear in Lehar’s work. 

Hence, on his materialist background, Lehar rejects direct 
(naive) realism which suggests that we can have experience of ob- 
jects out in the world directly, as if bypassing the chain of sensory 
processing. Provided that the materialist background is retained, 
I agree with this rejection. But if we apply an idealist worldview, 
our perceptions are of course experienced directly, and based on 
these perceptions we form concepts, such as the concepts of a 
“material” object, a “material” world, and perceptual models such 
as Lehar’s Gestalt Bubble model. I see these concepts and mod- 
els as mental constructs representing features of the perceptual 
reality, such as quantitative features and three-dimensional 
Gestalt features. These constructs are of course also experienced 
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directly, and they can be made unambiguous and precise. Here I 
agree with Lehar, who thinks that perceptual models remain 
“safely on the subjective side of the mind/brain barrier” (empha- 
sis in original) and writes about “objective phenomenology” lead- 
ing to “perceptual modeling” (sect. 4). It is when we accord “ma- 
terial” concepts a special existence of their own, principally 
different from the existence of conscious experiences, that is, 
when we move to materialism, that we run into trouble with di- 
rect realism. 

Lehar finds troubles with indirect realism as well but eventually 
accepts this view on the premise that the world we see around us 
is not the real external world but a miniature virtual-reality replica, 
an internal data structure within our physical brain. I think this 
view gives only an incomplete, imprecise conception of the “ex- 
ternal world,” including our “physical brain.” This incompleteness 
and imprecision are shared with other philosophies assuming in- 
direct realism, such as “hypothetical realism” (Léw 1984; Ran- 
drup, submitted; Wuketits 1984), “commonsense realism” (Ruse 
1986), and Kant’s concept of “the things in themselves” versus “the 
things for us.” According to Kant’s philosophy, we actually know 
nothing about things in themselves, except that they are supposed 
to exist. I think that this uncertainty or renunciation of knowledge 
compares unfavorably with the precision of the “material” con- 
cepts based directly on perceptual data in the idealist worldview. 

Another shortcoming of materialism in relation to the study of 
cognition is that it is difficult consistently to avoid dualism, as ap- 
pears from Lehar’s views about qualia mentioned above. And if 
dualism is admitted, it is hard to see how conscious experiences 
can be generated by material processes in the brain, as Lehar 
thinks they are (sect. 2.4). In the alternative idealist view of the 
world, it is not so hard to see, conversely, how “material” concepts 
are generated by the mind; the history of science shows how such 
concepts have been created (e.g., quanta, superstrings) or deleted 
(impetus, phlogiston, the ether) following the advent of new per- 
ceptual (observational) experiences. The special material type of 
existence is not a part of the idealist philosophy. (For a more ex- 
tensive discussion of the mind-matter and mind-brain problems 
in relation to cognition, see Knight 2001; Randrup 1997; 2002.) 

Actually I think that Lehar’s study, based on “the primacy of 
subjective conscious experience” and leading to a model of phe- 
nomenal perception, is most readily understood within the ideal- 
ist worldview, and within this view his troubles with direct and in- 
direct realism, with materialist monism, and with mind-matter 
relations will be significantly reduced. For more about the ideal- 
ist worldview proposed here, see Randrup (1997; 2002). 


Consciousness as phenomenal ether? 
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Abstract: The Gestalt Bubble model of visual consciousness is a coura- 
geous attempt to take the first-person perspective as primary in the study 
of consciousness. I have developed similar ideas as the Virtual Reality 
Metaphor of consciousness (Revonsuo 1995; 2000). I can, hence, only 
agree with Lehar about the general shape of a proper research strategy for 
the study of consciousness. As to the metaphysical basis of the research 
program, I have, however, several reservations about panexperientialism. 


I agree with Lehar on several points but disagree about the ulti- 
mate metaphysical nature of consciousness. I shall first describe 
points of agreement and then proceed to a criticism of panexperi- 
entialism. First, any research program on consciousness should 
start by taking the explanandum seriously, constructing a system- 
atic description of it. This is Lehar’s “objective phenomenology.” 
In the context of the biological sciences, this is the initial, de- 
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scriptive stage of inquiry. All branches of biology have begun with 
the descriptive stage, and the study of consciousness should be no 
exception. 

Second, in the study of consciousness the top-down approach 
should be of at least as much importance as the bottom-up ap- 
proach. Once we have a detailed description of the structure, or- 
ganization, and dynamics of a higher level of organization (in this 
case, subjective phenomenology), it will impose significant con- 
straints on the possible lower-level (neural) mechanisms that 
could account for the higher-level features. The lower-level mech- 
anism must be capable of supporting exactly the kind of structure, 
organization, and dynamics as is found at the higher level of phe- 
nomenology; otherwise the proposed mechanism is not a plausi- 
ble candidate to explain the phenomenon. The bottom-up strat- 
egy is important too, but it should be combined with the top-down 
strategy. Otherwise bottom-up approaches may lead either to the 
elimination of consciousness (because it is so difficult to see how 
single-neuron activity could add up to holistic features of con- 
sciousness), or to the search for the mere neural correlates of con- 
sciousness (rather than the directly underlying constitutive mech- 
anisms that explain the phenomenon), because the signals that are 
collected from the brain usually originate nowhere near the higher 
levels of organization where consciousness itself resides (Revon- 
suo 2001). 

Third, indirect realism as a theory of perception seems to be the 
only alternative that can give a plausible explanation of dreams and 
other hallucinations. Dream experiences show that the brain in 
rapid eye movement (REM) sleep can bring about a fully con- 
vincing simulation of the perceptual world and a simulated self 
embodied inside this virtual world. Dreams are temporally pro- 
gressing “being-in-the-world” experiences generated inside the 
brain. During dreaming, phenomenal consciousness is causally 
isolated from the stimulus environment, from the concurrent state 
of the physiological body, and from behavioral output systems. As 
I have argued in my previous Behavioral and Brain Sciences com- 
mentaries on Pessoa et al. (1999) and O’ Regan & Noé (2001), their 
theories of visual consciousness cannot account for our vivid visual 
experiences in dreams. 

Although I therefore largely agree with Lehar as to what the 
proper approach to the study of consciousness should be, there is 
one core issue on which we seem to have differing views. His fun- 
damental metaphysical commitment is to panpsychism (or panex- 
perientialism), according to which (a simple form of) conscious- 
ness is a fundamental property of physical matter. According to 
this view there is no radical discontinuity between any physical 
systems as to the possession of consciousness; it is just a matter of 
degree. Everything is more or less conscious; simple physical sys- 
tems to a lesser degree, the human brain perhaps to the highest 
possible degree. This smooth continuum of consciousness across 
all physical entities is supposed to have the following explanatory 
strengths: (1) consciousness is a fundamental property of physical 
matter and therefore need not be explained in terms of (noncon- 
scious) physical matter; (2) there is no radical conscious/noncon- 
scious dichotomy to be found anywhere in the natural order (e.g., 
in phylogeny or ontogeny). 

This approach raises some severe problems. There are clear, 
well-demonstrated dichotomies between the presence and the ab- 
sence of the state of consciousness (caused by anesthesia, epilep- 
tic seizures, fainting, coma) and between the presence and ab- 
sence of particular contents of consciousness even though the 
stimuli are implicitly processed (as in blindsight or neglect). Any 
theory of consciousness should be able to explain these radical 
subjective differences between the presence and absence of con- 
sciousness. The panexperientialist is, however, forced to say that 
these are not really cases where the presence and total absence of 
consciousness in the brain could be strictly contrasted. The con- 
trast is only between primitive and more sophisticated forms of 
consciousness. According to the panexperientialist, the primitive 
form may be something so simple that we would hardly recognize 
it as consciousness at all. Hence, what we thought was the total ab- 


sence of experience is actually the presence of a primitive form of 
experience; we just cannot recognize it as experience. 

Unfortunately, this move will not help us to understand the rad- 
ical contrast between the presence and absence of conscious ex- 
perience in the above cases. Regarding everything as conscious (to 
some degree) does not remove the radical conscious/noncon- 
scious contrasts. In fact it leads to a position as difficult as (but the 
exact opposite to) the eliminativist position defended by Dennett. 
If we take either the panexperientialist position that phenomenal 
consciousness is everywhere in the world or the eliminative posi- 
tion that it is nowhere, we are no closer to explaining the radical 
empirical differences that we want to understand. 

Furthermore, panexperientialism smacks of a misuse of the 
concept of experience. It is difficult to see why the postulated 
“primitive form” of consciousness — which we might not even rec- 
ognize as experience — should be placed in the same category as 
our vivid phenomenal experiences. There seems to be no clear 
idea of what “protoconsciousness” could be, whether it exists at all, 
or how the claims for its existence could be empirically tested or 
theoretically modeled; and how exactly the primitive form of con- 
sciousness relates to our ordinary, vivid, phenomenal conscious- 
ness. 

Hence, I do not regard panexperientialism as an advisable 
metaphysical commitment for a research program on conscious- 
ness. I would rather postulate that the sphere of subjective expe- 
rience is a higher level of biological organization in the brain. Phe- 
nomenal experience exists only at that level and in those creatures 
whose brains can realize that level. Otherwise, the physical uni- 
verse is devoid of phenomenal consciousness. When we totally 
lose consciousness, as we do during anesthesia, for example, our 
brain is temporarily incapable of supporting the phenomenal level 
of organization. The radical difference between the presence and 
the absence of phenomenal experience is to be described and ex- 
plained in terms of biological levels of organization in the brain. 
Physical matter at lower levels of organization perhaps may be said 
to contain the potentiality of being conscious, but only in the weak 
sense in which all physical matter contains the potentiality to be 
alive. The mere potentiality does not make simple physical sys- 
tems (say, carbon atoms or diamonds) alive, and it would be a 
waste of time to study the microphysical structure of diamonds in 
order to understand the biology of living systems. Ina similar vein, 
I fear that the assumption that all physical systems (diamonds, 
toothbrushes, bacteria, and so on) are conscious (or “protocon- 
scious”) is going to be a useless, untestable hypothesis for the sci- 
ence of consciousness. 

Protoconsciousness seems to be comparable to “ether,” the in- 
visible form of matter that was once believed to fill all physical 
space. The idea of a vacuum devoid of physical matter was 
unimaginable. Perhaps the idea of a “phenomenal vacuum” or the 
total absence of conscious experience is equally difficult to accept. 
But although there were genuine empirical phenomena that the 
ether models tried to account for, there seem to be no phenom- 
ena (either nonconscious physical or conscious phenomenal) that 
the phenomenal ether of panexperientialism accounts for. Fur- 
thermore, as far as we know there are total phenomenal vacuums, 
total absences of phenomenal experience, and we should not try 
to fill them by postulating a phenomenal ether that pervades all 
physical matter. Instead, our theories of consciousness should ex- 
plain the definitive differences, both phenomenal and biological, 
between the total presence and the total absence of consciousness 
in the brain. 
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Abstract: Lehar (rightly) insists on the volumetric character of our expe- 
rience of space. He claims that three-dimensional space stems from the 
functional three-dimensional topology of the brain. But his “Gestalt Bub- 
ble” model of volumetric space bears an intrinsically static structure — a 
kind of theater, or “diorama,” bound to the visual modality. We call atten- 
tion to the ambivalence of Gestalt legacy and question the status and pre- 
cise import of Lehar’s model and the phenomenology that motivates it. 


Lehar should be applauded for making a strong case for the fun- 
damental character of volume and depth in our experience of 
space. The originality of his proposal resides inter alia in the rad- 
ical claim that a three-dimensional experience of space stems from 
the functional three-dimensional topology sustained by the hu- 
man brain (not to be naively equated with brain topography). He 
posits that subjective spatial experience criterially requires a 
three-dimensional topological substratum — a device lacking a 
three-dimensional topological-dynamical structure could never 
account for the volumetric experience of space. In other words, 
the only viable option for a functionalist indifferent to brain phys- 
iology is three-dimensional topological-dynamical functionalism. 

Lehar depicts his model as an outgrowth of Gestalt tradition. 
Indeed, one can easily recognize two essential features of Gestalt 
theory: its phenomenological approach to subjective experience, 
and the postulate of psychophysical isomorphism. Phenomeno- 
logical space, its emergence, and its scientific explanation as a 
brain process are, according to Lehar, grounded in pregiven, con- 
tinuous, and coherent topology, specifically a three-dimensional 
functional topology. 

Lehar may not be aware that the way Gestalt psychologists 
treated space was in reality quite equivocal. Although they were 
in principle cognizant of the fundamental status of volumetric 
space, they granted it low priority in their scientific agenda and 
tended “provisionally” to treat space as a series of transparent/ 
opaque surfaces, if not as ambient ground against which to set a 
figure. On the other hand, it is true that Kohler’s theory of psy- 
chophysical isomorphism explicitly referred to three-dimensional 
functional brain topology to construe not only three-dimensional 
geometrical static structures but also two-dimensional structures 
evolving in time (see Koffka 1935). The theory combined empiri- 
cal and phenomenological constraints with speculative brain 
physics (e.g., the theory of cortical fields) so as to represent both 
brain process and phenomenological experience in a single dy- 
namical scheme (see Rosenthal & Visetti 2003). 

Several attempts have been made to model Gestalt principles 
of perception in accordance with neurophysiology and in par- 
ticular with the doctrine of neural coding (e.g., of perceptual 
microfeatures). For example, models of neural fields or neural 
repertoires feature a two-dimensional functional topology that 
corresponds to a topographic two-dimensional arrangement of 
units in primary areas (e.g., retinotopy) (e.g., Hoffman 1989; 
Koenderinck 1990; see Petitot 1999 for a review). Less discrete 
models, unconstrained by brain physiology, were developed in the 
context of image processing, and sometimes resorted to fairly 
complex mathematics, but maintained set to a bidimensionality of 
their input (retinal or pictorial; see Morel & Solimini 1995). The 
very idea of three-dimensional functional topology was hardly 
taken into consideration in the few attempts to account for depth 
(e.g., Grossberg 1994), which therefore had to resort to hosts of 
specialized coding units: a patently implausible solution, as Lehar 
rightly noted. 

The solution advocated by Lehar is original and certainly de- 
serves attention. He defines a three-dimensional topological mi- 
liew where any local element can be in one of four states (corre- 
sponding to local surface elements). Each individual element (or 
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point in a perceptual matrix) exerts a field influence on adjacent 
elements for them to take on a similar state (or to be prevented 
from this by inhibition). Reciprocal determination between sur- 
face elements is assumed to generate equilibrium in which the rel- 
evant features are stabilized. The input to the model is an image 
set in the frontal plane (much like a retinal image). The output (ac- 
tually the first step in “geometrization” of space) is a distribution 
of geometrical surface microfeatures in a three-dimensional 
space. Although Lehar does not mention this issue, one can read- 
ily deduce that unit formation or individuation is assumed to take 
place in this three-dimensional visual matrix. The originality of 
this proposal should be highlighted: Whereas the majority of rival 
models first individuate two-dimensional units (from two-dimen- 
sional image input), then categorize them as faces of three- 
dimensional units, Lehar sets his three-dimensional structure ab 
initio, and whatever is to populate this three-dimensional distri- 
bution of geometrical microfeatures supposedly comes next. 

It is not clear, however, which scientific question Lehar has set 
out to answer. He does not seem to attempt another perspectival 
reconstruction of the visual field, for his model, in contrast to its 
alleged purely phenomenological motivation, builds on a physi- 
calist metaphor. Although Lehar dismisses neurophysiological 
concerns, the analogy between his model and neural net models 
jumps to the eye: Traditional “neurons” with their receptor fields 
are replaced by elements or points in perceptual matrix, and 
neural connections are supplanted by fields of influence. More- 
over, Lehar alludes to the possibility that the model may take a dis- 
crete or granular form (see target article, Fig. 7A). Why, then, does 
he hammer so loudly his physicalist credo? It seems that Lehar be- 
lieves that the process by which space is constituted necessarily 
sheds light on the way we perceive space. Then why does he not 
try to motivate his model genetically? Clearly, Lehar needs to tell 
us the rules of the scientific game he plays more explicitly (does 
he want to model the constitution of space from a purely phe- 
nomenological viewpoint or does he attempt a free mathematical 
reconstruction of subjective experience?). 

Lehar could have mentioned that during the past century other 
theorists put forth elaborate proposals concerning the constitution 
of space experience (e.g., Gibson 1950; Husserl 1907/1997; Poin- 
caré 1905/2001). Instead of sticking to neurophysiology, they re- 
ferred to the structure of the organism or the lived body. These 
were strongly dynamic, sensorimotor “models” of constitution of 
phenomenological space that assumed a multimodal origin of vol- 
umetric space and explicitly related its dimensionality to reper- 
toires of self-generated movements. Although none of these 
“models” can be regarded as fully effective, they account for the 
ontogenesis of space in a dynamic fashion and for a variety of phe- 
nomena of adaptation (e.g., to distorting or inverting goggles). We 
suggest that considering the dynamics of the genetic, multimodal, 
and sensorimotor character of the constitution of space is as im- 
portant in modeling perceived space as neurophysiology and the 
kind of static geometry on which Lehar elaborates. What comes 
along with such dynamics is the constitutive relationship between 
external and bodily space. Lehar appears to be aware that per- 
ception of space involves one’s own body, but instead of taking this 
as a constitutive relation, he treats the body as just another object 
in space. 

Finally, we have strong reservations with respect to Lehar’s phe- 
nomenology. The field of vision he refers to neglects readiness for 
prospective action, and the phenomenological subject is not im- 
mersed in the practical field of ongoing activity with its qualita- 
tive, praxeological, and prospective dimensions (see Rosenthal & 
Visetti 2003). What about the nonisotropy of perceived space and 
the resulting potential heterogeneity in the constitution of regions 
of space? Is it advisable to consider phenomenological space as a 
mere deployment (be it three-dimensional), independent of the 
engaged or prospective actions to which it gives stage? 
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Abstract: Lehar argues that a simple Neuron Doctrine cannot explain 
perceptual phenomena such as size constancy but he fails to discuss exist- 
ing, more complex neurological models. Size models that rely purely on 
scaling for distance are sparse, but several models are also concemed with 
other aspects of size perception such as geometrical illusions, relative size, 
adaptation, perceptual learning, and size discrimination. 


Lehar argues (sect. 2.2 and elsewhere) that there are no adequate 
neurological models to explain why we see the world the way we 
do, and that theorists have ignored the discrepancies between the 
proximal stimulus and our perceptual experience. He then pre- 
sents a computational model to describe our perceptual experi- 
ence of hyperbolic space. He rightly complains about the shortage 
of neurological models for size and shape constancy but he fails to 
discuss the models that do exist. 

Psychologists have long been interested in size scaling, or dis- 
crepancies between perceived size and image size: The phenom- 
ena include size constancy, geometrical illusions, optical distor- 
tions, adaptation, and aftereffects. The classical account of size 
constancy maintains that size is scaled for distance in a quasi-geo- 
metric manner (the size-distance invariance hypothesis); this ac- 
count is not productive of neurological models because it assumes 
that retinal image size is “correctly” encoded in the visual cortex 
and that the image is then scaled for distance in some unexplained 
“cognitive” manner. Kirschfeld (1999) argues that the image rep- 
resentation has to be scaled for distance neurologically before it 
enters consciousness and that this might be done in area V4. He 
notes that Dobbins et al. (1998) found that some neurons in this 
area varied their response to the angular size of lines depending 
on viewing distance. 

The idea that image size is transformed at some preconscious 
stage of visual processing by mechanisms other than distance scal- 
ing (e.g, McCready 1985) may be more fruitful. Stuart et al. 
(1993) proposed a computational model based on broadly tuned 
layers of size detectors, which could account both for Weber's law 
in size discrimination and for the biasing effects of geometrical il- 
lusions; however, they did not extend the model to include scaling 
for distance. The main alternative approach to size constancy — 
generally supported by Gibsonians — is that object sizes are scaled 
in relation to the surrounding spatial scale. This approach has the 
advantage of embracing other size illusions in addition to size con- 
stancy and it is more productive of neurological models. Size con- 
trast illusions have been attributed to adaptation of cells that de- 
tect spatial frequency or to other neural interactions in the brain 
(see Gillam 1998). However, spatial frequency is not the same 
thing as image size (the distance across an image), so spatial fre- 
quency models are unhelpful for general models of size percep- 
tion. 

Andrews (1964) proposed a perceptual learning model of size 
calibration in which the brain corrects the metric of the visual field 
according to the most recent information and attempts to equal- 
ize the spacing of contours. This would allow for learning in addi- 
tion to explaining some illusions, aftereffects, and size constancy. 
Richards (1977) suggested that simple cells in the cortex might re- 
spond to relative rather than absolute size and he also discussed 
the properties necessary for the neural basis of size constancy. 

Some authors have attempted to explain size constancy through 
the enlargement of perceived size for the central part of the visual 
scene, which occurs because the representation of the central part 
of the retinal image covers more cortical cells at later stages of 
analysis. Such an idea is based on the anatomical fact of cortical 
magnification, which enhances acuity for central vision. The fovea 
contains more densely packed cone cells than the surrounding 
area and it projects to a relatively larger region of the primary vi- 


sual cortex. Schwartz (1980) incorporated this idea into his model 
of size constancy. When an observer fixates a distant object, it 
forms a small image in central vision, whereas close objects form 
larger images that spread further into the periphery: The small 
central image is therefore expanded neurologically relatively more 
than the larger image. Such a mechanism might contribute mar- 
ginally to size constancy, but it fails to explain how objects of the 
same angular size can appear different in size even when both are 
viewed in central vision. 

An example of this problem is the moon illusion (see Ross & 
Plug 2002). The moon illusion is the apparent enlargement of the 
sun or moon when low on the horizon compared with its size when 
higher in the sky; the effect is similar to size constancy but is hard 
to explain by the usual “scaling for distance” account. The diffi- 
culty is that the low moon appears nearer than the high moon, 
whereas size-distance invariance requires it to appear further. 
Trehub (1991, pp. 242-47) developed the “retinoid” model, 
which could account for both size constancy and the moon illu- 
sion. He argued that size magnification is expensive in neurologi- 
cal terms because it involves the use of more networks of cells. The 
brain husbands its resources by magnifying only the most “eco- 
logically relevant” parts of the scene — that is, objects in the near 
distance when looking horizontally, and close overhead when 
looking up. Humans cannot normally interact with celestial ob- 
jects or with distant terrestrial objects, so the images for such ob- 
jects can safely be left relatively small. Size constancy is therefore 
poor for far horizontal distances and even poorer when looking 
upwards. The three-dimensional representation of distance is also 
shrunk vertically in comparison with horizontally, again for the 
purpose of minimizing neural resources. Distance is computed 
within the three-dimensional retinoid system and is represented 
by “sheets” of cells; the extent of size magnification is linked to the 
distance plane onto which the image is mapped. This biased map- 
ping of the visual scene onto brain structures is largely the result 
of human evolution, but it can be further modified by individual 
experience. 

There are neuropsychological findings that support multiple 
representations of three-dimensional space (see Previc 1998). 
There are also findings on micropsia and hemineglect that give 
clues as to how and where size might be coded (see Kassubek et 
al. 1999). Lehar may be correct that a simple Neuron Doctrine 
cannot account for size scaling, but more complex neurological 
models show promise. 
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Abstract: Collapsing three-dimensional space into two violates Lehar’s 
“volumetric mapping” constraint and can cause the visual system to con- 
struct illusory transparent regions to replace voxels that would have con- 
tained illumination. This may underlie why color constancy is worse in two 
dimensions, and argues for Lehar to revise his phenomenal spatial model 
by putting “potential illumination” in empty space. 


Lehar’s phenomenological description of space neglects the fact 
that empty space is actually full of illumination. For example, if a 
cast shadow crosses half of this page and you move your finger 
from a word under shadow to one under full illumination, you are 
not surprised when your finger crosses the shadow, even though 
your finger is closer in depth than the page. This is because every 
voxel between your eye and the page contains some amount of 
light. It is unfortunate that Lehar overlooks this fact, because he 
correctly asserts that depth information is volumetric, whereas 
current neurological models fail to “represent transparency], ] 
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with multiple depth values at every single (x,y) location, or to rep- 
resent the experience of empty space between the observer and a 
visible object” (sect. 3, para. 5). These same models also ignore 
that every voxel of “empty” space contains light of some intensity 
and chromaticity. 

This confusion probably results from naively accepting the pop- 
ular notion that humans care only about the location and qualities 
of objects, making the perception of illumination irrelevant. This 
assumption is so prevalent that much of color research is devoted 
to determining how the visual system “discounts the illuminant.” 
However, a viable solution to the Gestalt problem of color con- 
stancy will emerge only with a more complete description and un- 
derstanding of how we subjectively experience illumination. Lron- 
ically, Lehar’s aspiration to describe the subjective experience of 
spatial vision in terms comparable to those of color vision reveals 
that current color vision research is also in peril. That is, he claims 
that color phenomena are reducible to hue, intensity, and satura- 
tion because that is how the brain represents them physiologically 
(sect. 2.3). Yet models of hue, intensity, and saturation cannot be 
the “primitives of raw conscious experience” (sect. 4, para. 3), in 
that these qualities remain invariant as illumination changes 
across space. 

This confound is apparent when Lehar discusses his Figure 1 as 
containing “explicit volumes, bounded by colored surfaces, em- 
bedded in a spatial void” (sect. 5.1, para. 2), where “every point 
can encode either the experience of transparency or the experi- 
ence of a perceived color at that location” (sect. 6). His more ac- 
curate intuition is that there are also intermediate states between 
transparent and opaque “to account for the perception of semi- 
transparent surfaces” (sect. 8.1, para. 1). I suggest that Lehar con- 
sider filling these semitransparent voxels with “potential illumina- 
tion” “at multiple depth values at every single (x,y) location.” This 
would also strengthen his second and third conclusions that “vol- 
umes of empty space are perceived with the same geometrical fi- 
delity as volumes of solid matter” and that “multiple transparent 
surfaces can be perceived simultaneously” (sect. 10, points 2 and 
3). Having semitransparent voxels contain “potential illumination” 
is a more parsimonious description of the void between your eyes 
and this page. You can actualize the “potential illumination” of 
these voxels by placing your finger in front of any shadow cast on 
the page. More accurately, Lehar’s phenomenological model al- 
lows only the plane of voxels directly in front of a given surface to 
contain cast shadows (i.e., less illumination), because the voxels 
that compose the surface must be the color of the opaque surface 
itself (sects. 5.1, 8.1). 

Note that this concept is not merely peripatetic (Aristotle 1976) 
or an ether explanation, in that we are always subjectively aware 
of the illuminant. For example, by looking from your illuminated 
reading room into a dark hallway, your subjective experience is not 
only that the hallway walls are under less illumination but also that 
the space itself contains less light. In this way, “potential illumina- 
tion” can also address why color constancy differs in two- versus 
three-dimensional scenes. For example, Gilchrist (1977) had ob- 
servers look through a pinhole into a room containing a doorway 
into a second room. The near room was dimly illuminated and the 
far room was highly illuminated. Attached to the door frame were 
several papers, arranged so that a mid-gray paper appeared either 
adjacent to the door frame or (with its corners removed) on the 
far room’s back wall. The lightness of the paper shifted in the di- 
rection of lightness constancy depending on whether it appeared 
on the door frame or on the far wall. Schirillo et al. (1990) gener- 
ated equivalent stimuli in two dimensions using a stereoscopic 
cathode ray tube (CRT) and stereoscope, yet this replication pro- 
duced only a fraction of Gilchrist’s constancy. I hypothesize that 
this occurred because stereoscopic space does not contain the ac- 
tual voxels of either high (e.g., near room) or low (e.g., far room) 
illumination. In essence, Schirillo and colleagues failed to pre- 
serve Lehar’s necessary condition of “volumetric mapping” (target 
article, Fig. 1D). 

The ubiquitous use of CRT images reduces scenes to Alberti’s 
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window, which retains perspective cues but eliminates Lehar’s re- 
quirement that space be volumetric. This obfuscates the color 
constancy paradox, in that these voxels contain illumination. For 
example, Adelson’s (1993) famous wall-of-blocks illusion contains 
cubes of identical luminance that appear dissimilar in lightness 
and concomitantly under illusory transparent stripes. Logvinenko 
et al. (2002) eliminated both the appearance of transparency and 
the lightness illusion by constructing a three-dimensional version 
of Adelson’s two-dimensional display. I hypothesize that the visual 
system does not add a transparent veil to Logvinenko’s display be- 
cause it already ascribes illumination to every voxel in space. How- 
ever, when Adelson eliminates such volumes but retains the same 
spatial geometry via X-junctions, the visual system reconstructs 
the volume to contain regions of illusory transparency (ie., illu- 
mination). Consequently, Lehar’s improved spatial model requires 
a phenomenal description of empty space that includes “poten- 
tially illuminated” voxels. 


If vision is “veridical hallucination,” what 
keeps it veridical? 


Peter Ulric Tse 

Department of Psychological and Brain Sciences, Dartmouth College, 
Hanover, NH 03755. Peter.Tse@dartmouth.edu 
http://www.dartmouth.edu/artsci/psych/faculty/tse.html 


Abstract: If perception is constructed, what keeps perception from be- 
coming mere hallucination unlinked to world events? The visual system 
has evolved two strategies to anchor itself and correct its errors. One in- 
volves completing missing information on the basis of knowledge about 
what most likely exists in the scene. For example, the visual system fills in 
information only in cases where it might be responsible for the data loss. 
The other strategy involves exploiting the physical stability of the environ- 
ment as a reference frame with respect to which the eyes and body can 
move. 


[S]pace and time are only forms of sensible intuition, and hence 
are only conditions of the existence of things as phenomena. . . 
we can have no cognition of an object, as a thing in itself, but 
only as an object of sensible intuition, that is, as phenomenon 

—Immanuel Kant (Critique of Pure Reason, 1781) 


Lehar develops the Kantian insight that perception is (1) entirely 
a mental construction; (2) lacks access to the world-in-itself to de- 
termine the accuracy of its representations; and (3) is only possi- 
ble given an internal framework of space-time that permits sen- 
sory input to be interpreted as occurring in an external space-time. 
Here I focus on how the brain can construct true information 
about the world when there is no way to judge objectively whether 
that information is true by comparing that information to the 
world-in-itself. 

To create veridical information, the visual system must com- 
pensate for errors, data loss, and processing bottlenecks imposed 
by its imperfect design. It has nothing but the ambiguous, incom- 
plete, and noisy image to determine whether it has made an error. 
It must therefore know what types of image cues indicate errors 
and it must have strategies to correct errors. The visual system cor- 
rects itself only when it is responsible for errors or data loss. It 
compensates for its own likely errors using two strategies. One is 
to rely on world knowledge, and the other is to assume that the 
world is stable. 

For example, when does the visual system fill in missing phe- 
nomenal features and when does it merely note that completion 
takes place without filling-in (see Fig. 1)? Filling-in occurs when 
the information that is missing from the image is missing because 
of the visual system’s own failure to detect it. The visual system fol- 
lows the principle “no news isn’t necessarily bad news when there 
was no way to get the news in the first place.” The visual system 
functions as if it knows that it does not always have adequate in- 
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Figure 1 (Tse). When (a) is replaced all at once by (b) a smooth 
apparent motion (indicated by an arrow) is filled in (Tse & Logo- 
thetis 2002). No filling-in occurs in (d) cases of amodal comple- 
tion (Tse 1999b). The shape behind the occluder, whether (e) or 
(f), is not completed. 


formation in a particular domain to determine the structure of the 
world. Hence, when information is missing from an image, this is 
not necessarily regarded as contradictory information or informa- 
tion that the undetected thing does not exist in the world. The in- 
formation might be undetected because of poor viewing condi- 
tions or because of the inherent limits on detection imposed by a 
noisy perceptual apparatus that has limited sensitivity. A more pre- 
cise formulation is: 

In the absence of direct (but presence of appropriate indirect) image 

evidence for the existence of x, under viewing conditions where x would 

not be detectable in the image even if it were present in the world, the 
visual system may not only not reject x, it may assume x to be the case, 
and interpolate x so that x is seen as if it were visible. 

Filling-in occurs because the visual system in effect blames it- 
self. The sensitivity of the visual system under given viewing con- 
ditions can be too poor to permit detection of an entity that indi- 
rect image evidence implies exists. Under such conditions the 
visual system creates what it “knows” must be missing. In amodal 
completion there is no filling-in because the visual system does not 
blame itself. The shape or features of the occluded portions of an 
object are not filled in, because under no possible viewing condi- 
tions would shape or surface features be visible through an opaque 
occluder. No matter how insensitive the visual system might be, it 
cannot blame itself for not detecting entities that are in principle 
undetectable under any viewing conditions. 

A second strategy to overcome potential errors is to analyze i im- 
age data under the assumption that the world is stable. First, the 
visual system does not need to store detailed information about 
the world because it can always sample the world for more infor- 
mation (O’Regan 1992). Second, the visual system can stabilize 
perceptual space by relying on the presumed stability of the world. 
For example, the retinal image usually changes en masse only 
when the body or eyes move. The system can exploit this stability 
in order to maintain the eyes and body in a constant position with 
respect to the world. A classic demonstration of this is the “mov- 
ing room” experiment (Lee & Aronson 1974), in which a person 
stands in a room that is set on rollers. When the walls move, rather 
than assume that the world has moved, the visual system assumes 
that the body has moved and corrects for this by changing the 
body’s position. Sometimes subjects even fall over. It is as if the vi- 
sual system blames itself for the discrepancy caused by the mov- 
ing room and compensates by relying on a world that it wrongly 
assumed was stable. 

Another example can be found in the recalibration of percep- 
tual space that takes place after a saccadic eye movement. Deubel 
and colleagues (Deubel et al. 1998) have argued that the system 


seeks its saccade target immediately after a saccade. If this target 
is found within a certain spatial and temporal window, the visual 
system assumes the target object to have remained stable and uses 
it as a reference object to determine the positions of other objects. 
This is true even when the target object in fact moves during the 
saccade. Even more surprisingly, Deubel and colleagues find that 
if the target has moved to the right, and a neighboring distractor 
has not moved at all, the visual system creates a percept of a tar- 
get that has remained stationary and a distractor that has jumped 
to the left. Because the visual system’s initial saccade lands accu- 
rately on the position where the target was at the beginning of the 
saccade, the visual system should know that the target has changed 
position. But this is true only if it assumes its saccade was infalli- 
bly correct. Instead, a corrective saccade is automatically made to 
the new position of the target, and the object is assumed to have 
remained stable. The distractor’s illusory jump to the left is filled 
in because it is the motion that must have occurred, assuming the 
stability of the target and the world. Again, it is as if the visual sys- 
tem blames itself for the discrepancy and relies on the stability of 
the world to correct its presumed error. 

Because the visual system has no direct access to the world, it 
must rely solely on the image to judge whether it has made errors 
in specifying the image-to-world correspondence. Error correc- 
tion is only possible based on assumptions about world structure 
and statistics. Completion may be phenomenal or not, depending 
on whether the visual system “blames itself” for the data loss. In 
addition, the visual system takes a world that it assumes to be sta- 
ble as its frame of reference. These two strategies allow the visual 
system to overcome the handicap that the truth of perceptual in- 
formation cannot be judged by comparing that information with 
the world-in-itself. 


Is the world in the brain, or the brain 
in the world? 


Max Velmans 

Department of Psychology, Goldsmiths College, University of London, New 
Cross, London SE14 6NW, United Kingdom. m.velmans@gold.ac.uk 
http://www.goldsmiths.ac.uk/departments/psychology/staff/ 
velmans.html 


Abstract: Lehar provides useful insights into spatially extended phenom- 
enology that may have major consequences for neuroscience. However, 
Lehar’s biological naturalism leads to counterintuitive conclusions, and he 
does not give an accurate account of preceding and competing work. This 
commentary compares Lehar’s analysis with that of Velmans, which ad- 
dresses similar issues but draws opposite conclusions. Lehar argues that 
the phenomenal world is in the brain and concludes that the physical skull 
is beyond the phenomenal world. Velmans argues that the brain is in the 
phenomenal world and concludes that the physical skull is where it seems 
to be. 


Is the phenomenal world in the brain, or is the brain in the phe- 
nomenal world? As William James (1904) noted, “the whole phi- 
losophy of perception from Democritus’s time downwards has 
been just one long wrangle over the paradox that what is evidently 
one reality should be in two places at once, both in outer space and 
in a person’s mind.” James defended the former view, and conse- 
quently developed a form of neutral monism in which the phe- 
nomenal world can be regarded as being either “mental” or “phys- 
ical” depending on one’s interest in it. If one is interested in how 
the appearance of the perceived world depends on perceptual 
processing, one can think of it as mental (as a psychological effect 
of that processing). If one is interested in how some aspect of the 
perceived world relates to other aspects of that world (e.g., via 
causal laws), one can think of it as physical. Lehar, by contrast, de- 
fends “biological naturalism” (a form of “physicalism’”) — the view 
that the experienced world is literally in the brain. 
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The difference is fundamental. But whatever view one takes 
about where to locate the perceived world, one fact is clear: the 
three-dimensional world that we see around our bodies, which we 
normally think of as the “physical world,” is part of conscious ex- 
perience, not apart from it. This perceived world is related to the 
unperceived world described by physics (in terms of quantum me- 
chanics, relativity theory, etc.) but it is not identical to it. This is 
potentially paradigm-shifting for the reason that it redraws the 
boundaries of consciousness to include the perceived physical 
world, with consequences for our understanding of mind-body re- 
lationships, subjectivity versus objectivity, science, epistemology, 
and much else (see extensive discussions in Velmans 1990; 1991a: 
1991b; 1993; 1996; 2000; 2001; 2002a; 2002b). As Lehar notes in 
his target article, this conceptual shift also has consequences for 
neurophysiology. An accurate phenomenology of consciousness is 
a prerequisite for an adequate understanding of the neural 
processes that support that phenomenology. In this, Lehar’s 
Gestalt Bubble model provides an interesting, original, and po- 
tentially useful step forward. 

Given the fundamental nature of the issues and the positive 
contributions of his paper, it is a pity that Lehar’s review of pre- 
ceding and competing positions is often inaccurate and unneces- 
sarily dismissive. For example, I barely recognised my own work 
on these problems from his summary. I do not have space to cor- 
rect these errors here,! but merely note that apart from a few cru- 
cial differences, Lehar’s understanding of the consciousness-brain 
relationship in visual perception is virtually identical to my own. 

What are the crucial differences? Consider the simple model of 
visual perception shown in Figure 1. Viewed from the perspective 
of an external observer E (not represented within the figure), light 
rays reflected from an entity in the world (which E perceives to be 
a cat) stimulate S’s eye and visual system. Neural representations 
of the entity, including the neural correlates of consciousness, are 
produced in S’s brain. In terms of what E can observe that is the 
end of the story. However, once the conditions for consciousness 
form in S’s brain, she also experiences a cat out in the world — so a 
full story of what is going on has to combine what E observes with 
what S experiences (see discussion of mixed-perspective explana- 
tions in Velmans 1996; 2000). If we combine EF’s observations with 
those of S, an entity in the world (the initiating stimulus) once 
processed, is consciously experienced to be an entity in the world 
(a cat), making the entire process “reflexive.” 

But here’s the puzzle: The neural representations of the cat (ob- 
served by E) are undoubtedly in S’s brain, so how can S experience 
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Figure 1 (Velmans). 
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the cat to be outside her brain? The effect is natural and ubiqui- 
tous, so there must be a natural explanation. Lehar’s Gestalt Bub- 
ble model gives some indications of what is achieved but does not 
suggest how itis done — and at present, we just do not know. How- 
ever, both virtual reality and holography might provide useful 
clues (Pribram 1971; Revonsuo 1995; Velmans 1993; 2000). Sup- 
pose, for example, that the information encoded in S’s brain is 
formed into a kind of neural “projection hologram.” A projection 
hologram has the interesting property that the three-dimensional 
image it encodes is perceived to be out in space, in front of its two- 
dimensional surface, provided that it is viewed from an appropri- 
ate (frontal) perspective and it is illuminated by an appropriate 
(frontal) source of light. Ifthe image is viewed from any other per- 
spective (from the side or from behind), the only information one 
can detect about the image is in the complex interference patterns 
encoded on the holographic plate. In analogous fashion, the in- 
formation in the neural “projection hologram” is displayed as avi- 
sual, three-dimensional object out in space only when it is viewed 
from the appropriate, first-person perspective of the perceiving 
subject. And this happens only when the necessary and sufficient 
conditions for consciousness are satisfied (when there is “illumi- 
nation by an appropriate source of light”). When the image is 
viewed from any other, external perspective, the information in S’s 
“hologram” appears to be nothing more than neural representa- 
tions in the brain (interference patterns on the plate). 

The “projection hologram” is, of course, only an analogy, but it 
is useful in that it shares some of the apparently puzzling features 
of conscious experiences. The information displayed in the three- 
dimensional holographic image is encoded in two-dimensional 
patterns on a plate, but there is no sense in which the three-di- 
mensional image is itself “in the plate.” Likewise (contra Lehar), I 
suggest that there is no sense in which the phenomenal cat ob- 
served by S is “in her head or brain.” In fact, the three-dimensional 
holographic image does not even exist (as an image) without an ap- 
propriately placed observer and an appropriate source of light. 
Likewise, the existence of the phenomenal cat requires the par- 
ticipation of S, the experiencing agent, and all the conditions re- 
quired for conscious experience (in her mind/brain) have to be 
satisfied. Finally, a given holographic image exists only for a given 
observer and can be said to be located and extended only where 
that observer perceives it to be.? S’s phenomenal cat is similarly 
private and subjective. If she perceives it to be out in phenome- 
nal space beyond the body surface, then, from her perspective, it 
is out in phenomenal space beyond the body surface. 

But this does not settle the matter. To decide whether the phe- 
nomenal cat is really outside S’s head, we have to understand the 
relation of phenomenal space to physical space. Physical space is 
conceived of in various ways depending on the phenomena under 
consideration (e.g., as four-dimensional space-time in relativity 
theory, or as eleven-dimensional space in string theory). However, 
the physical space under consideration here and in Lehar’s analy- 
sis is simply measured space. Lehar agrees, for example, that at 
near distances, phenomenal space models measure space quite 
well, whereas at far distances this correspondence breaks down 
(the universe is not really a dome around the earth). How do we 
judge how well phenomenal space corresponds to measured 
space? We measure the actual distance of an object within phe- 
nomenal space, using a standardised measuring instrument — at 
its simplest, a ruler — and count how often it has to be placed end 
to end to get to the object. Although rulers look shorter as their 
distance recedes, we know that their length does not significantly 
alter, and we conclude therefore that distant objects are really fur- 
ther than they seem. 

Lehar and I agree (with Kant) that whether we are “subjects” 
or “external observers,” we do not perceive things as they are in 
themselves — only phenomena that represent things themselves, 
and, together, such phenomena comprise our personal phenome- 
nal worlds. In Figure 1, for example, the cat, the subject's head, 
and the neural representations in S’s brain (as they appear to E) 
are as much part of E’s phenomenal world as the perceived cat is 
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part of S’s phenomenal world. This applies equally to rulers or 
other instruments that E might use to measure distance. In sum, 
to carry out his science, E does not have an observer-free view of 
what is going on, anymore than S does. E and S simply view what 
is going on from different third- and first-person perspectives. 
This has extensive consequences (worked out in Velmans 2000), 
but I have space to comment on only one of these here. Accord- 
ing to Lehar, the three-dimensional phenomenal world in my own 
analysis is “undetectable externally by scientific means,” does not 
“exist in any true physical sense,” and is therefore “a spiritual en- 
tity to be believed in (for those who are so inclined), rather than 
anything knowable by, or demonstrable to, science” (target article, 
sect. 2.3, para. 8). Nothing could be further from the truth. Data 
in science consist entirely of observed phenomena that occur in a 
spatially extended phenomenal world, and the measurements that 
we make within that phenomenal world are the only ones we have 
on which to ground our science! 

Where is this phenomenal world? Viewed from E’s perspective, 
it is outside his head, and the distance of the phenomenal objects 
within it can be measured, using standardised instruments that 
operate on phenomenal space (the distance of this phenomenal 
page from your eye, for example, can be measured with a ruler). 
Viewed from E’s perspective, the phenomenal world also appears 
to be represented (in a neural form) in S’s brain. Viewed from S’s 
perspective, things look the same: The phenomenal world appears 
to be outside her head, and, if she looks, a neural representation 
of that world appears to be encoded in E’s brain. Given that the 
evidence remains the same, irrespective of the perspective from 
which it is viewed, one can safely conclude (with James) that al- 
though a neural encoding of the world is within the brain, the phe- 
nomenal world is outside the brain. As this is how the natural 
world is formed, there must be a natural explanation (see above). 
I have shown (Velmans 2000) how this analysis can be developed 
into a broad “reflexive monism” that is consistent with science and 
with common sense. 

Now consider Lehar’s alternative: It is widely accepted that ex- 
periences cannot be seen in brains viewed from the outside, but 
Lehar insists that they can. Indeed, he insists that E knows more 
about S’s experience than S does, and S$ knows more about E’s ex- 
perience than E does, as the phenomenal world that S experiences 
outside her brain, is nothing more than the neural representation 
E can see inside her brain, and vice versa. This has the conse- 
quence that the real physical skull (as opposed to the phenomenal 
skull) exists beyond the phenomenal world. As Lehar notes, the 
former and the latter are logically equivalent. 

Think about it. Stick your hands on your head. Is that the real 
physical skull that you feel or is that just a phenomenal skull in- 
side your brain? Ifthe phenomenal world “reflexively” models the 
physical world quite well at short distances (as I suggest), it is the 
real skull, and its physical location and extension are more or less 
where they seem to be. If we live in an inside-out world, as Lehar 
suggests, the skull that we feel outside our brain is actually inside 
our brain, and the real skull is outside the farthest reaches of the 
phenomenal world, beyond the dome of the sky. If so, we suffer 
from a mass delusion. Our real skulls are bigger than the experi- 
enced universe. Lehar admits that this possibility is “incredible.” 
I think it is absurd. 


NOTES 

1. Details are given in an unabridged version of this commentary (Vel- 
mans 2003). 

2. The position of the image relative to the plate, for example, changes 
slightly as the observer moves around the plate. Nevertheless, the image 
is sufficiently clear for the observer to (roughly) measure its width and how 
far it projects in front of the plate (e.g., with a ruler). 


Percepts are selected from nonconceptual 
sensory fields 


Edmond Wright 


3 Boathouse Court, Trafalgar Road, Cambridge CB4 1DU, United Kingdom. 
elw33@hermes.cam.ac.uk http://www.cus.cam.ac.uk/~elw33 


Abstract: Steven Lehar allows too much to his direct realist opponent in 
using the word “subjective” of the sensory field per se. The latter retains its 
nonconceptual, nonmental nature even when explored by perceptual 
judgement. He also needs to stress the evolutionary value of perceptual dif- 
ferences between person and person, a move that enables one to under- 
mine the direct realist’s superstitious certainty about the singular object. 


With regard to the title of Steven Lehar’s article, it is vital that the 
term “subjective” not be used of the structurally isomorphic sen- 
sory field. To acquiesce in its use is to yield ground to the direct 
realist opponent. It can be credibly argued that the ground of all 
sensory experience is thoroughly nonconceptual, beyond that of 
Gareth Evans's use of the term, in that recognisable entities and 
properties are not given in the initial stage of the process, not even 
that of a subject (Evans still took “nonconceptual” to include the 
perception of separable objects-as-unrecognised; Evans 1982, 
p. 228). The isomorphic field, because of its very isomorphism, is 
as brute as the input at the sensory organ, therefore as nonmental 
as material (Wright 1996), whatever its nature may be as an emer- 
gence from complexity. How could it not be if it is, however indi- 
rectly, covariant with the input? As John Foster has put it, sensing 
is something that just “happens to us” rather than “something that 
we do” (Foster 2000, p. 123). Subjectivity does not enter into the 
equation until the establishment of perceptual judgement and 
memory has taken place upon that nonconceptual evidence at the 
behest of the motivational module. Therefore, it is going too far to 
attribute “protoconsciousness” at this level (sect. 6.5), for this cor- 
rection regards sensing as always existing apart from judgement, 
merely evidence upon which a mind may or may not work (Wright 
1996, pp. 24-28). 

Lehar (sect. 2.4) justifiably uses the analogy with the television 
screen employed by Roy Wood Sellars, Barry Maund, and Virgil 
C. Aldrich (Sellars 1916, p. 237; Maund 1975, pp. 47-48; Aldrich 
1979, p. 37), in that the distinction made between the screen-state 
(of the phosphor cells) and what is judged to be shown upon it is 
structurally similar to that between the sensory evidence within 
the brain and the percepts chosen from it. If he accepts the co- 
gency of this comparison, then he ought to acknowledge that the 
radically nonconceptual nature of the sensory evidence is implied 
by this analogy. However much information-theoretic evidence 
there may be on screen/neural raster, it registers only covariations 
with light-wave frequencies and intensities at the camera/retina, 
not any information about recognisable entities and properties (if 
the TV set was upside down and one had just entered the room 
where it was, one would be unable to use one’s memories to judge 
that, say, lan McKellen as Gandalf was at that moment “visible,” 
the screen thus revealing its permanently nonconceptual state). So 
Lehar should accept the criticism made above. 

Those anti-qualia philosophers and psychologists who inveigh 
against the “picture-in-the-head” proposal (e.g., O'Regan & Noé 
2002), have always opposed the television analogy. Lehar does not 
sufficiently defend himself against this attack (sect. 2.3). As [have 
pointed out (Wright 1990, pp. 8-11), there cannot literally be pic- 
tures in the head, for, if colours are neural events, actual pictures 
are not coloured, and the “picture” in the head is. Nor is an eye re- 
quired for sensing neural colour, for eyes are equipped to take in 
uncoloured light-waves, and there are no light-waves in the head. 
Visual sensing is a direct experience for which eyes would be use- 
less. Gilbert Ryle’s attempt to maintain that one would have to 
have another sensation to sense a sensation remains as an argu- 
ment, as Ayer described it, “very weak” (Ryle 1949/1966, p. 203; 
Ayer 1957, p. 107). 

Once this radically nonconceptual nature of the fields is admit- 
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ted, its evolutionary value can be brought out, which is precisely 
what Roy Wood Sellars and Durant Drake — the very philosophers 
that Lehar calls to his aid — insisted upon (target article, sect. 2.3; 
Drake 1925; Sellars 1922). Sellars particularly stressed the feed- 
back nature of the perceptual engagement, which allows for the 
continual updating of entity selection from the fields (altering spa- 
tiotemporal boundaries, qualitative criteria, etc.), a claim that ren- 
ders stances such as Gibson’s which take the object as given (amus- 
ingly termed “afforded”; Gibson 1977), not so much as “spiritual,” 
the term favoured by Lehar (sect. 2.3), but as literally superstitious. 

What weakens the direct realist case is its unthinking reliance 
on the pre-existing singularity of “external things.” Ifthe feedback 
argument of Sellars pére is correct (Sellars 1970, p. 125), then the 
perfectly singular “object” or “entity” is but a feature of the mode 
of perceiving and not ontological in its nature. The behaving as if 
it is singular, the trusting assumption that it is, is a necessary fea- 
ture of the intersubjective cooperation, for we could not even 
roughly coordinate our differing percepts unless we did project a 
strictly imaginary perfectly common focus of them; but it is fatal 
to take the convergence as without residue, for that would cancel 
the possibility of feedback and hence of mutual correction. 

Lehar adverts to the uncertainty of the object (sect. 6.1). The 
only basic ontology required under the theory above is of the ma- 
terial continuum: When human social perceiving is in operation, 
with its incessant intersubjective correction in action, then a very 
modest ontological further claim can be made, namely, that a com- 
munity of correctors exists, and hence of selves and their sensory 
fields, but not as fixed entities, only as current tentative selections 
from sensory and motivational experience. The direct realist, by 
contrast, is committed to an indefinite number of separable sin- 
gular entities (objects and persons), a superstition that is discon- 
certingly all too common in recent books on the philosophy of per- 
ception (from Millar 1991 to Thau 2002; there are very few 
exceptions, e.g., Maund 1995). The act of faith in singularity which 
is necessary to bring our differing percepts into some kind of 
working overlap, is taken by the direct realist as actual, which rep- 
resents an insidious and dangerous move to the conviction that his 
own percept is the standard for all. 
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paradigmatic assumptions 
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Abstract: To avoid endless and futile debate, critics of an alter- 
native paradigmatic hypothesis cannot simply state their own par- 
adigmatic assumptions as if they were plain fact while dismissing 
those of the opposition as self-evidently absurd, because it is ex- 
actly those initial assumptions that are brought into question by 
the paradigmatic proposal. Perceived incredibility is no valid 
ground for rejection of a paradigm whose alternatives are at least 
equally incredible, and arguably more so. 


The energetic responses of the open peer commentaries in- 
dicate that the target article has touched a raw nerve; this 
is perhaps a harbinger of an interesting direction of inves- 
tigation. The epistemological issue at the core of the debate 
is a paradigmatic question that challenges some of the foun- 
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dational assumptions of psychology and neuroscience, 
which have remained so long unchallenged that they are 
generally held to be established fact. As is frequently the 
case in paradigm debates, the opposing camps often cite the 
selfsame evidence to support their opposite conclusions, 
because they are arguing from different foundational as- 
sumptions. To avoid endless debate, it is therefore essential 
for commentators to recognize the paradigmatic issue at 
the core of the debate, and not just state their own para- 
digmatic assumptions as if they were established fact — 
while dismissing those of the opposition as self-evidently 
absurd — because it is exactly those initial assumptions that 
are brought into question by the target article. If alternative 
paradigms are to be fairly evaluated, it is necessary to tem- 
porarily and provisionally suspend one’s own paradigmatic 
assumptions, and accept the assumptions of the alternative 
paradigm as if they could actually be true. Only then can the 
competing paradigms be fairly compared, not on the basis 
of the perceived incredibility of their initial assumptions, 
but on the overall coherence and self-consistency of the 
world view they implicate in total. 


R1. Rigor paradigmatis 


Unfortunately, many of the commentators failed to grasp 
the paradigmatic nature of the proposal and restated their 
own paradigmatic assumptions as if they were plain fact, 
thus committing the error of petitio principii, assuming 
from the outset that which is to be proven. 

Booth complains that it is “foolish” to look for con- 
sciousness among the brain cells. I contend that it is foolish 
to look for it anywhere else but in the brain! As in most par- 
adigmatic debates, one man’s “foolish” is another man’s “ob- 
vious.” But Booth says not a word about the epistemologi- 
cal difficulties, which were discussed at length in the target 
article, of the view that he defends. If the experience of a 
red surface, for example, is located anywhere else but in the 
brain, then it is a spatial structure that exists, but it does not 
exist in any space known to science. This makes Booth’s hy- 
pothesis a religious or spiritual theory, because the experi- 
enced surface is in principle beyond detection by scientific 
means, and therefore it is a theory that is impossible to dis- 
prove. It’s no good trying to dismiss the structure of con- 
sciousness in a trick of grammar, as Booth proposes, by 
claiming that the spatial structure of experience is a “seem- 
ing” rather than something real. That objection was ad- 
dressed in the target article with the observation that visual 
consciousness has an information content, and information 
cannot exist independent of an actual physical mechanism 
or substrate in which it is registered. Booth seems to think 
that simply stating his own paradigmatic hypothesis as if it 
were plain fact (“We are not looking at a world inside our 
minds; we are . . . seeing the colour of the patch out there.”) 
is an adequate response to the hypothesis that what we are 
seeing really is in our brain. 

Dresp complains that I fail to make clear the link be- 
tween the Gestalt Bubble model and general theories of 
consciousness. 


What the model has to do with consciousness . . . remains to- 
tally unclear. Neither the fact that we are able to consciously ex- 
perience and describe three-dimensional shapes as entities and 
wholes, nor the fact that we can find laws or codes describing 
how these emerge perceptually, implies or proves that con- 
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sciousness is necessary to see and move around in three-di- 
mensional space. 


The link to general theories of consciousness is through the 
philosophy of identity theory (Feigl 1958; Russell 1927), as 
explained in the target article, whereby mind and brain are 
not separate and distinct but are ontologically one and the 
same. The presence of spatial structures in the perceptual 
representation is identically equal to a conscious experience 
of those structures. 

Dresp simply assumes as if it were plain fact, that the ex- 
periential component of consciousness is separate and dis- 
tinct from the physical mechanism by which it is instanti- 
ated, and that, therefore, a model of the mechanism cannot 
possibly be a model of the experience, because the experi- 
ence cannot be modeled in principle. But it is at least 
equally likely prima facie that experience is not separate 
and distinct from the mechanism that carries it, but that ex- 
perience is a physical process taking place in the physical 
brain, so a model of the mechanism would automatically 
also be a model of the experience. In fact, this is by far the 
more parsimonious explanation because it employs a single 
explanans, the brain, to account for the properties of mind 
and brain. Identity theory is an equally valid paradigmatic 
alternative that cannot be dismissed without demonstrating 
why it is less credible than a mystical, nonphysical theory of 
experience beyond science. Furthermore, the Gestalt Bub- 
ble model is explicitly defined as a model of experience, 
rather than its neurophysiological correlate, so it is incon- 
ceivable how Dresp fails to see the connection between a 
model of experience and the experience it models. 

Dresp objects to my exhortation to discover the real 
truth behind visual processing. “Who said that science has 
to bother with metaphors such as ‘truth’?” she asks. But sci- 
ence is all about modeling objective external reality, a truth 
that science presupposes to exist. Either there are “pictures 
in the head,” as explicit spatial structures, or there are not. 
And whether there are pictures in the brain is of primary 
importance for psychology, philosophy, and neuroscience. 
Simply defining those pictures as a mystical nonphysical en- 
tity brings us no closer to understanding how consciousness 
arises in brains. 

Duch asks: “How can the physical skull encompass non- 
physical, inner world? “The world inside the head’ is a 
metaphor, and it does not make much sense to invert it, un- 
less one believes that there is some kind of physical world 
squeezed inside the skull.” That is precisely the hypothesis 
I presented, although it is a perceptual world, expressed in 
physical form, which is squeezed inside the skull. This no- 
tion seems to Duch so absurd from the outset that he can- 
not believe that is what I am proposing. Duch states, as if it 
was plain fact, that “interpretation of the spatial structure of 
the states of the visual system has nothing to do with their 
physical location.” This is exactly the issue brought into con- 
tention by the target article. If he is to contest this assump- 
tion, he must explain which incredible alternative he wishes 
to defend in its place. Does he claim that consciousness does 
not exist at all, as Dennett (1981) would have us believe? Or 
does he allow that it exists but in some hidden dimension in- 
accessible to scientific scrutiny, as Velmans proposes? If so, 
how does he address my critique that his view of conscious- 
ness is a religious rather than a scientific hypothesis? 

In his conclusion, Duch says “It is doubtful that the 
Gestalt Bubble model can explain observations that have 
not been hidden in its premises.” That is the nature of par- 


adigmatic hypotheses, and is just as true of the hypothesis 
that Duch defends. If you begin from the outset with the 
assumption that consciousness has no location, then you are 
guaranteed never to find it located anywhere! 

Fox begins his commentary with the complaint that 
“{Lehar] ignores much of what is known in perceptual and 
brain science.” The truth is that Lehar challenges much of 
what is “known” in perceptual science. Far from ignoring, 
I have taken pains to point out the errors of what is known 
in perceptual and brain science. If Fox begins with the as- 
sumption that those supposed “facts” are indisputable, then 
he is bound to have trouble coming to grips with a hypoth- 
esis that they may perhaps be mistaken. 

There is much in Fox’s commentary that is deeply mys- 
terious. Fox accuses me of maintaining the “Cartesian 
mind-body distinction.” But the central hypothesis of the 
target article is an identity theory whereby mind is nothing 
other than the functioning of the physical brain. This monis- 
tic view is diametrically opposed to Cartesian dualism. Fur- 
thermore, in section 2.3, I explicitly refute Cartesian dual- 
ism as a spiritual rather than a scientific hypothesis. 

Laming also fails to grasp the paradigmatic nature of the 
proposed model, presenting as counterarguments the ax- 
ioms of his own paradigmatic alternative as if they were 
plain facts, rather than unsubstantiated initial assumptions. 
Laming insists that some parts of visual experience can be 
shared with others while the remainder are private, and 
therefore there cannot be a natural science of perception. 
So a psychophysical report that, for example, a subject can 
or cannot see an extended red object, is a valid subjective 
report. But the redness and spatial extendedness of that 
perceived object are not validly reportable because they are 
private. Curiously, the very aspects of experience that Lam- 
ing considers illegitimate, including all of the Gestalt prop- 
erties surveyed in sections 5 and 7 of the target article, are 
exactly the aspects of experience that reveal the spatial 
structure that Laming insists have no physical reality. Lam- 
ing must explain why the spatial aspect of perception is so 
private that it cannot be reported, when the Gestalt Bubble 
model clearly demonstrates how the spatial aspects of per- 
ception can be reported and quantified in a spatial model. 
If he contests my phenomenology and claims not to see the 
sky as a dome, and the sides of a road converge to a point, 
then he should tell us what he sees instead. It is a paradig- 
matic choice of Laming’s, not a statement of incontestable 
fact, to call the spatial extendedness of perception unre- 
portable. And if Laming chooses to believe that phenome- 
nal consciousness is not a physical entity in the brain, he 
should address the clear objections to that paradigm out- 
lined in the target article. In particular, conscious experi- 
ence, according to Laming, is a spatial structure; it is a 
structure that exists, and yet it does not exist in any space 
known to science, and it is in principle undetectable by sci- 
entific means. This is a religious or spiritual hypothesis be- 
cause it is impossible in principle to disprove. To accept 
Laming’s view of consciousness is to declare consciousness 
in principle forever beyond explanation in scientific terms 
— which would then become a self-fulfilling prophecy. 


R2. Perceived incredibility 


Some commentators reject the representationalist thesis 
because they find it to be frankly incredible. Velmans 
complains 
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Stick your hands on your head. Is that the real physical skull that 

you feel or is that just a phenomenal skull inside your brain? . . . 

If we live in an inside-out world, as Lehar suggests, the skull 

that we feel outside our brain is actually inside our brain, and 

the real skull is outside the farthest reaches of the phenomenal 
world, beyond the dome of the sky .. . Our real skulls are big- 
ger than the experienced universe. Lehar admits that this pos- 
sibility is “incredible.” I think it is absurd. 
And with this, I believe, Velmans touches on the principal 
reason this alternative has been given so little considera- 
tion. 

I am viscerally sympathetic with this objection, so much 
so that for years I too refused to accept the conclusion to- 
ward which all of the evidence points. It is indeed incredi- 
ble to think that your physical head is larger than the dome 
of the sky. But science has discovered many things which 
were initially considered to be at least equally incredible; 
like the vastness of the universe, and its cataclysmic gene- 
sis from a singularity in space and time, and the bizarre na- 
ture of black holes and of quantum phenomena. All of these 
theories were initially held to be incredible but have since 
been accepted as established fact. And the reason they were 
accepted is not that they became any less incredible. Sci- 
entific fact is accepted on the basis of the evidence, re- 
gardless of the incredible truth to which that evidence 
points. In fact, that is exactly what gives science the power 
to discover unexpected or incredible truth: When the obvi- 
ous explanation is blocked by chronic paradoxes, it is time 
to give the seemingly incredible alternative a serious look. 

In his conclusion, McLoughlin invokes Occam’s razor 
against the notion of the world of experience being a pic- 
ture inside the head. I believe McLoughlin reflects the con- 
sensus view in neuroscience: that the hypothesis seems 
frankly too incredible to deserve serious consideration. But 
before deploying Occam’s razor we must first balance the 
scales and take a full accounting of the alternatives under 
consideration. For the alternative is that experience is a spa- 
tial structure; it is a structure that exists, but it exists in a 
separate space that is inaccessible to scientific scrutiny: It is 
a structure with a vast information content, but with neither 
mass nor energy nor spatial presence in the physical uni- 
verse known to science, and the brain conducts a continu- 
ous two-way exchange of information with this phenome- 
nal, semiexistent nothing. Alternatively, McLoughlin might 
prefer Dennett's (1981) eliminative alternative, that con- 
scious experience, the spatial structure under discussion, 
simply does not exist — and that’s the end of the problem of 
consciousness. If McLoughlin finds the idea of the world- 
in-your-head incredible, he must balance his rejection of it 
by telling us which of the other two incredible hypotheses 
he finds more credible, otherwise it is his bubble that 
bursts, not mine. 


R3. Paradigmatic alternatives 


Not every challenge to alternative paradigmatic hypotheses 
is a case of rigor paradigmatis. It is perfectly valid to chal- 
lenge a paradigm based on the overall coherence or self- 
consistency of the world view that it implicates in total. In 
fact that is the only way that paradigms can be fairly com- 
pared, as seen in the commentaries discussed in this sec- 
tion. 

Revonsuo accepts the representationalist view of the 
target article but challenges panexperientialism. Revonsuo 
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cites phenomena, such as neglect and blindsight, that sug- 
gest perceptual information can be processed without nec- 
essarily entering consciousness. But, as in the case of most 
paradigm debates, both camps can usually muster an ex- 
planation for almost any phenomenon raised, although 
each explanation is consistent only within its own paradigm 
and sounds patently absurd from the point of view of the 
other. 

For example, blindsight, the apparently unconscious 
processing of visual information, can be explained as a case 
of amodal perception. When the blindsight patient reports 
a vague suspicion of motion in the absence of an experience 
of a moving object, he is reporting a conscious experience 
of a vague suspicion of motion without actually seeing any- 
thing in motion. Even people with normal vision commonly 
have such experiences in peripheral vision, and many psy- 
chophysical experiments measure vague perceptual experi- 
ences at the threshold of detection which are not so much 
seen as suspected to have possibly appeared. Many philoso- 
phers deny that amodal percepts, or other forms of non- 
sensory knowledge, can be validly considered as conscious 
processes, insisting instead that only modal experience is 
experience. But if we exclude the amodal component of 
perception from conscious experience, then by definition, 
amodal perception is always blindsight. It seems more par- 
simonious to suppose that amodal perception is consciously 
experienced, even if only amodally, than to suppose that 
something experienced can be unconscious. 

Other examples of apparently unconscious processing 
can be explained in the panexperiential view as separate, 
parallel, and largely independent conscious processes in the 
brain. The part of the brain that performs the processing is 
indeed conscious of its own performance, but it is not in 
touch with the part of the brain that reports on that pro- 
cessing, So no processing is reported. Similarly, unconscious 
processing in dichotic listening can be explained by sepa- 
rate, parallel streams of consciousness, one of which over- 
writes the other, which is therefore never recalled. This is 
not to say the evidence of neglect and blindsight favors pan- 
experientialism, merely that it does not refute it. The moti- 
vation for panexperientialism lies elsewhere. 

It is true, as Revonsuo suggests, that the protocon- 
sciousness hypothesis probably cannot make testable pre- 
dictions, but that is not why I invoked it in the target arti- 
cle. It was raised to plug up some otherwise serious holes in 
a purely monistic or physicalist explanation of conscious ex- 
perience. This paradigmatic choice avoids a most subtle 
residual dualism hidden in Revonsuo’s alternative. 

As long as a sharp step or abrupt discontinuity is allowed 
between conscious and unconscious processes, there will 
always be an explanatory gap, because physically, the brain 
can be disassembled into ever-smaller pieces, all the way to 
atoms and molecules; whereas in Revonsuo’s view, con- 
sciousness does not have this ability to be disassembled but 
disappears abruptly as soon as the minimal conditions for it 
are no longer met. To be clear, I do not dispute that con- 
sciousness may exhibit, and indeed appears to exhibit, an 
abrupt cut-off — for example, when falling asleep or waking 
up — although intermediate semiconscious states are also 
known. If consciousness appears abruptly at some level of 
organization, then something else physically observable 
must come into existence at that point also. 

For example, consciousness might be identified with a 
holistic process such as spatial standing waves of electro- 
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chemical resonance in the brain (Lehar 2003). Resonance 
shares with consciousness the property of coming abruptly 
into being when the conditions for its formation are just 
right, as when blowing a note in a musical instrument. And 
yet, standing waves are not some supervenient spiritual en- 
tity but a real, physically measurable phenomenon that 
emerges holistically in a physical system. But if standing 
waves were indeed the physical substrate of consciousness, 
then that would suggest that musical instruments also nec- 
essarily possess some form of primal spatial consciousness. 
And when global consciousness breaks down in the brain, 
whether as a result of sleep or of anesthesia, the global syn- 
chrony does not disappear so much as it breaks up into a 
million fragments of locally isolated coherence. Would 
these fragments not each experience an isolated fragmen- 
tary consciousness? If not, then we have again an abrupt 
discontinuity that suggests a dualism between experience 
and its physical correlate. The more likely alternative is that 
parallel fragmented states of consciousness are indeed ex- 
perienced during periods of unconsciousness but they 
never register in memory and are therefore quickly forgot- 
ten, as are many dreams. 

A further problem with the abrupt discontinuity of con- 
sciousness is that it opens the problem of the “bridge locus” 
in the brain; that is, the question of why some very special 
parts of the brain become conscious, while other parts do 
not. It also leads to problems with partial or fragmented 
consciousness, as in split-brain patients, and in cases of mul- 
tiple personality syndrome, and hypnotic or trance states, 
which all seem to indicate multiple parallel or alternating 
consciousnesses in a single brain. Whether the disassem- 
bled pieces of consciousness can be usefully considered 
conscious in any real sense is admittedly a semantic ques- 
tion. But what is not semantic is the question of whether 
consciousness can be disassembled into component pieces, 
as the matter and energy in the brain can be, or whether the 
mind operates by different laws than the matter that is 
sometimes its physical correlate. The only self-consistent 
physicalist explanation is that complex consciousness in a 
complex brain occurs by the same principle as simple con- 
sciousness in simpler brains; and the same argument prop- 
agates all the way down to the root of the phylogenic tree 
and beyond. Consciousness is what it feels like for matter 
to exist, and complex human consciousness is what it feels 
like to be the waves of energy resonating in a human brain. 

Revonsuo complains that the panexperientialist position 
brings us no closer to explaining the radical empirical dif- 
ferences that we want to understand. Quite to the contrary: 
Until we bring consciousness fully into the realm of the 
physical world, one small corner of it will remain perma- 
nently trapped in a supervenient dimension forever, in prin- 
ciple, beyond scientific scrutiny. That is the modern “ether” 
theory that must be shown to be pure vacuum. 

Gunderson observes that visual experience consists of 
more than just a spatial structure — itis a spatial structure 
that is experienced as being viewed from a particular point 
— and that this aspect of viewing from a point is not cap- 
tured in the Gestalt Bubble model. In the first place, 
whether experience is viewed from a point or not, the fact 
remains: Visual experience is a spatial structure, and the 
spatial structure of experience is captured in the Gestalt 
Bubble model. 

But in fact, the Gestalt Bubble model goes further, sug- 
gesting that the experience as if viewing from a point is it- 


self an illusion. Once we recognize the world of experience 
for the internal model that it is, it becomes evident that our 
objective nouminal “self” is not the body-image homuncu- 
lus observed at the center of our phenomenal world, but 
that in fact, the whole world of experience is part of our real 
self. The blue of the sky is not observed “from” the ego- 
centric point, but it is experienced to exist out there where 
it lies at the surface of our perceptual sphere. The blueness 
of that azure dome is experienced to exist at a location rel- 
ative to the egocentric point, but it is in no sense transposed 
or projected back to the egocentric point. In fact, phenom- 
enally speaking, there is nothing special at all at the location 
of the egocentric point, which is experienced as an empty 
void of phenomenal space inside the phenomenal head, just 
like any other empty space in the phenomenal world. 

Three factors contribute to the vivid illusion of viewing 
the world from a point. The first is the body-image ho- 
munculus, which we take to be our real “self” because that 
particular piece of the phenomenal world is under our di- 
rect volitional control. Under large doses of hallucinogenic 
drugs such as LSD the perceptual distinction between self 
and nonself tends to disappear, as the body image merges 
with the background, leaving the entire sphere of percep- 
tual experience to be identified as “self,” a common theme 
also of Buddhist phenomenology. The second contributory 
factor is the warped geometry of phenomenal space that is 
organized around a center, the point of highest perceptual 
resolution, marking that center of symmetry as a special lo- 
cation in phenomenal space. Finally, the illusion is bol- 
stered by perceived surfaces being perceived modally only 
when they are exposed to the egocentric point, as if they 
were indeed being viewed from or by the egocentric point. 
Objects not exposed to the egocentric point are invisible to 
direct modal experience, and are therefore experienced in 
amodal fashion. 

A similar phenomenon is observed on a radar scope, 
where radar “echoes” are registered only from those sur- 
faces exposed to the central radar dish, for example, from 
the exposed front faces of nearby mountains. No echoes are 
registered from the hidden rear face of the mountains, nor 
from more remote surfaces occluded by the nearer moun- 
tains. As in perception, the center ofa radar scope is not the 
“observer” of the rest of the image on the scope, and ap- 
pears to be special only because the image on the scope is 
a veridical manifestation of the external, “nouminal” situa- 
tion, where the radar echoes are indeed received or 
“viewed” from the location of the radar dish — which is why 
no echoes are received from surfaces that are not exposed 
to that point. Similarly, the phenomenal experience of view- 
ing from a point is a veridical manifestation of the external 
nouminal situation where physical light from the external 
world is indeed received by the nouminal eye, no light be- 
ing received from hidden or occluded surfaces, creating the 
illusion that the phenomenal world is being viewed from 
the location of the phenomenal eye. 


R4. Reliability of phenomenology 


A number of commentators challenged the reliability of 
phenomenological observation. McLoughlin pointed out 
that naive observers are surprised to discover that they have 
a fovea and amazed that they have a blind spot in each eye. 
True enough, but the same naive observers can be easily 


BEHAVIORAL AND BRAIN SCIENCES (2008) 26:4 433 


Response/Lehar: Gestalt isomorphism and the primacy of subjective conscious experience 


educated by the most convincing demonstration of all, phe- 
nomenological observation of their own loss of resolution in 
peripheral vision and of their own blind spots. Like any tool, 
phenomenology is useful only if employed with intelli- 
gence. 

Booth complains that I commit the epistemological fal- 
lacy of trying to build public knowledge on the basis of pri- 
vate impressions. Booth objects that phenomenological ob- 
servations are private, and so “they cannot be wrong — but 
then neither can they be right.” But then he objects that my 
observations on phenomenal perspective are wrong! If phe- 
nomenological observation cannot be wrong, then how can 
Booth claim that my observations of phenomenal perspec- 
tive are wrong? Booth complains that it is impossible to look 
one way down a road and at the very same moment be look- 
ing the other way, So it cannot be correct to write that the 
two sides of the road must be bowed, as in Figure 2 of the 
target article. In the first place, itis not necessary to be look- 
ing in opposite directions at the same time to see the cur- 
vature of the phenomenal world, all one needs to do is to 
look in one direction and observe that the parallel sides of 
a road meet at a point at a distance which is less than infi- 
nite, and that those parallel sides appear straight and paral- 
lel throughout their length. And in the opposite direction 
one sees exactly the same thing, and in between one sees a 
spatial continuum exactly as depicted in Figure 2. Phe- 
nomenological observation can indeed be right and it can 
be wrong, and Booth’s phenomenology is just plain wrong 
if he can’t see perspective foreshortening in the three-di- 
mensional world around him! 

Hochberg suggests an alternative, less holistic model — 
a stage set rather than an all-encompassing bubble, with an 
abrupt discontinuity at a certain depth, where a proximal 
percept of a full three-dimensional road with perfectly par- 
allel sides changes abruptly to a flat two-dimensional expe- 
rience at right angles to the view direction, in which the 
sides of the road converge to a point in the plane of the 
backdrop. No matter how hard I try, I cannot see the world 
this way; I always see the two extremes of near and far per- 
ception seamlessly connected through a continuous inter- 
mediate zone, wherein the sides of the road are perceived 
in full three dimensions, and yet they are also perceived to 
converge, and they are perceived to be parallel even as they 
converge. 

I acknowledge that perception is somewhat more frag- 
mented than the Gestalt Bubble model suggests. For ex- 
ample, every visual saccade presents a momentary experi- 
ence perhaps somewhat like a stage set. But the most salient 
and immediate aspect of conscious experience is the way 
these individual theater sets are welded together into a uni- 
fied sphere of spatial experience. Whatever direction we 
gaze, we are constantly aware of where that gaze is directed 
in the global sphere of surrounding space, and the objects 
perceived in that direction are perceived to be located in 
that part of global space. The experience is more that of a 
stable, structured surrounding space than a series of theater 
sets showing successively on the same stage. 

As evidence to disqualify the Gestalt Bubble model, 
Hochberg cites visual illusions that vary as a function of 
where they are attended, because the same Gestalt is in 
view wherever it is attended. There are two aspects of spa- 
tial experience that must be carefully distinguished, we 
might call them global and focal. In the global experience 
our view is of a perfectly stable surrounding world, as sug- 
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gested in the target article’s Figure 2, whose entire surface 
is painted in modal colors, because whichever direction we 
look, that is the way it appears. The other aspect of experi- 
ence is focal, the immediate experience of looking in a par- 
ticular direction. The world appears at higher resolution in 
the direction of gaze than in the periphery, and the rear 
hemisphere behind our head is blank, as suggested in Fig- 
ure 15. Both global and focal aspects are observed in our ex- 
perience, so they should both be represented in a model of 
that experience. The combined experience is modal and fo- 
cal in the direction of sight, but amodal in the hidden rear 
portion of the field, as indicated by the dashed lines in Fig- 
ure 15, although successive saccades in different directions 
create an illusion of the complete modal sphere suggested 
in Figure 2. 

The point of the Gestalt Bubble model is not to deny that 
there are localized focal processes active in perception, but 
merely to add that there is also a single, globally unified per- 
ceptual experience, and the localized focal experiences are 
perceived to be embedded at specific locations in this larger 
global framework of spatial experience. 

Hochberg also cites examples of ambiguous or unstable 
percepts attributable to figures such as Adelson’s (Adelson 
2000) Impossible Staircase (Figure 2C of Hochberg’s com- 
mentary). Hochberg argues that since these percepts are 
observed to be unstable and/or ambiguous, they would dis- 
qualify a Gestalt Bubble model of a globally unified per- 
ceptual world. But Hochberg need not have gone farther 
than Figures 3,5, 6C, 11D, 12A, and 16A (porthole illusion 
variant) of the target article for examples of unstable, semi- 
stable, or ambiguous figures. The perceptual tendency to- 
wards a unified, globally consistent percept is a goal that the 
perceptual mechanism seeks, but does not always achieve, 
so unstable and multistable percepts are not counterexam- 
ples to the principle of emergence as described in the 
Gestalt Bubble model; they provide a more detailed look at 
the mechanism of that emergence. 

Hoffman argues that the perceived world of the Gestalt 
Bubble model is not a veridical replica of the external 
world, but merely a useful “user interface” to the external 
world, with no more need to resemble that world than a 
Windows interface needs to resemble the diodes, resistors, 
and software of a computer. Therefore, there may be no 
real resemblance at all between the structure of our phe- 
nomenal world and the real external world that it repre- 
sents. Hoffman proposes to replace indirect realism with 
this species-specific “user-interface” theory of perception. 
But that is exactly what I have proposed in the Gestalt Bub- 
ble model. Nowhere was it stated that the phenomenal 
world is in any sense identical to the external world. Phe- 
nomenal colors are very much more impoverished than the 
chromaticity of physical light, and phenomenal perspective 
shrinks the infinite external world into a finite bounded 
bubble. These are clearly species-specific “user-interface” 
simplifications of external reality. The “realism” in “indirect 
realism” is already modulated by the word “indirect,” that 
is, the phenomenal world is a very real and direct view of 
processes taking place within our own brain, and those 
processes in turn represent indirectly the structures and 
surfaces presumed to be present in the more remote exter- 
nal world. 

Immanuel Kant (1781/1991) anticipated Hoffman’s ob- 
servation that the phenomenal world need not show any re- 
semblance to the external physical world. We do not even 
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knowifit has three spatial dimensions and time; all we know 
is that those are the dimensions of the internal phenome- 
nal world. It should also be noted, however, that in every 
other realm of human activity, from hunting and gathering, 
to business and finance, to politics and engineering, the as- 
sumption that the world of experience is an accurate rep- 
resentation of objective reality is so successful that the in- 
directness of perception can be readily ignored. And that in 
turn suggests that experience must accurately reflect some 
essential aspects of the external, although Hoffman is right 
that we cannot determine phenomenologically which as- 
pects of the world are veridically replicated and which are 
not. 

Lloyd presents a very clever argument by analogy that 
appears to punch a hole through the central premise of iso- 
morphism. Consider the statement from the target article: 
“The fact that the world around us appears as a volumetric 
spatial structure is direct and concrete evidence for a spa- 
tial representation in the brain” (sect. 5.2, para. 6). Lloyd 
suggests that the absurdity of this statement can be revealed 
by substituting “colored” for “spatial” in this passage. Phe- 
nomenal color experience defines a three-dimensional re- 
lational structure of phenomenal color space. But, Lloyd 
correctly objects, the fact that we experience phenomenal 
color does not mean that the color solid appears anywhere 
in our brain, and by the same token, spatial experience does 
not imply a spatial structure in the brain. 

This objection is already addressed in the target article 
by the specification of information content as the one quan- 
tity which is necessarily preserved across the mind/brain 
barrier. A color experience is indeed a three-dimensional 
relational structure, not so different in principle from the 
color phosphor dots on a television monitor. It takes three 
values of three phosphor dots to encode a single point of 
color on your screen, and those three values define a single 
point in Red-Green-Blue [RGB] space. However, a single 
point on a television screen does not define an entire color 
solid, but merely one point in that three-dimensional color 
space, representing the color currently represented at that 
point. The color solid is not explicitly present anywhere, but 
the relational structure that it encodes is implicitly present 
in the range of possible values of the three phosphor dots. 

Spatial perception is different from color perception in 
this one significant aspect, that every point in perceived 
space can be perceived with a distinct color. That means 
that there are as many separate and potentially distinct 
color values in a perceived surface as there are resolvable 
points across that surface. The points in a perceived surface 
are perceived simultaneously and in parallel, and together 
they define a relational structure in which every point bears 
a specific spatial relation to every other point in that per- 
ceived surface. This is quite different from the implicit 
structure of color space that encodes only one color at a 
time, because spatial perception encodes a whole spatial ar- 
ray of color values, all of which are simultaneously present 
in experience. 

Regarding the value and prospects for phenomenology, 
Markovieé says that “without the precise specification of the 
extraphenomenological aspects of perception, such as the 
stimulus and neural domains, it is difficult to answer the 
question related to why the percept looks as it does.” This, 
however, is difficult only if one employs phenomenology 
merely to confirm theories of vision based on neurophysi- 
ology. Once we realize that what we are seeing in experi- 


ence is the representation in our own brain, there is a great 
deal that can be learned about why things look the way they 
do, and how things are represented in the brain. Markovié 
is right that scientific explanation must go beyond mere de- 
scription. In Markovié’s example, the Earth’s motion be- 
comes comprehensible only when considering the influ- 
ence of the Sun. But before science can propose 
explanations it must begin with description. The influence 
of the Sun on the Earth’s motion would have never become 
clear had we not first observed and described that motion. 
Psychology too must begin with a description of experience 
before it can attempt a plausible explanation for it. 


R5. Explicit volumetric representation 


McLoughlin points out that a volumetric space can be ex- 
pressed in a sparse, more symbolic code, without recourse 
to an explicit spatial array, with objects represented as to- 
kens, with x, y, and, location, and so forth. There are many 
aspects of mental function, such as verbal and logical 
thought, that are clearly experienced in this abstract man- 
ner. But visual consciousness has an information content, 
and that content is equal to the information of a volumetric 
scene in an explicit volumetric representation. Every point 
in the volume of perceived space is experienced simultane- 
ously and in parallel. To propose that the representation un- 
derlying that experience is a sparse symbolic code is to say 
that the information content of our phenomenal experience 
is greater than that explicitly expressed in the neurophysio- 
logical mechanism of our brains. 

Velmans’ holographic analogy is very apt. There is in- 
deed no “picture” as such on a holographic plate, just a fine- 
grained pattern of interference lines. But for the picture to 
be experienced by a viewer, or to be available for data ac- 
cess in an artificial brain, that picture must first be reified 
out of that pattern of interference lines into an actual im- 
age again; that is, the holograph must be illuminated by a 
beam of coherent light. After passing through the holo- 
graphic plate, that beam of light generates a volumetric ar- 
ray of patterned light, every point of which is determined 
by the sum of all of the light rays passing through that point, 
and it is that volumetric pattern of light in space that is ob- 
served when viewing a hologram. 

So if holography is to serve as a metaphor for conscious- 
ness, the key question is whether the metaphorical holo- 
gram is illuminated by coherent light to produce a volu- 
metric spatial pattern of light or whether the hologram in 
experience is like a holographic plate in the dark. Ifit is the 
former, then conscious experience in this metaphor is the 
pattern of light waves interfering in three-dimensional 
space. Itis a spatial image that occupies a very specific por- 
tion of physical space, and it requires energy to maintain it 
in that space. This is exactly the kind of mechanism we 
should be looking for in the brain. If it were the latter, as 
Velmans suggests, then why would the shape of our expe- 
rience not be that of the interference patterns etched on the 
holographic plate, rather than the volumetric image they 
encode? What magical substance or process in conscious 
experience performs the volumetric reconstruction that in 
the real universe requires an actual light beam and some 
complicated interference process to reconstruct? If it is a 
spatial structure that we observe in consciousness, then it is 
a spatial structure that we must seek out in the brain, not a 
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potentially spatial structure that remains stillborn in a non- 
spatial form. Otherwise, the spatial image-like nature that 
is so salient a property of subjective experience must re- 
main a magical mystical entity forever in principle beyond 
the reach of science. 

In the target article, 1 commended Grossberg for advo- 
cating explicit filling-in to account for Gestalt illusions, but 
chided him for not extending that same reasoning into the 
third dimension. To this, Grossberg responds that I have 
not kept up with the modeling literature; he cites the FA- 
CADE and LAMINART as models that explain many 
three-dimensional figure-ground, grouping, and filling-in 
percepts, including transparency, and that use an explicit 
surface filling-in process. This I do not doubt. But in both 
FACADE and LAMINART, depth is handled in a disparity 
based representation with left and right eye image pairs and 
disparity images to represent depth information. In neither 
of those models is there a three-dimensional volumetric 
spatial matrix with receptive fields at every location and 
every orientation in three dimensions, as would be required 
for a neural network model of spatial experience. While FA- 
CADE and LAMINART do perform explicit filling-in of 
both contours and surfaces, the filling-in itself does not 
propagate in the third dimension by diffusion as it does in 
the other two. The third dimension is handled very differ- 
ently than the other two, and the result is a 2-D sketch 
rather than a full volumetric spatial matrix. Whatever their 
merits as neurophysiologically plausible models, these 
models leave something to be desired as perceptual mod- 
els, because perceptual experience is fully volumetric and 
three-dimensional, and multiple depth values can be expe- 
rienced in any direction. 

If Grossberg’s argument that explicit filling-in is re- 
quired to account for two-dimensional illusions has any va- 
lidity at all, then it should apply just as well to three-di- 
mensional perception as it does to two, at least for a 
perceptual model that models the experience rather than its 
neurophysiological correlate. 


R6. World as external memory 


McLoughlin endorses O’Regan’s (1992) concept of seeing 
as an active process of probing the environment as though 
it were a continuously available external memory. But prob- 
ing the world with visual saccades, especially in the mon- 
ocular case, is nothing like accessing a memory, internal or 
external, because every saccade retrieves only a two-di- 
mensional pattern of light. The three-dimensional spatial 
information of the external world is by no means immedi- 
ately available from glimpses of the world but requires the 
most sophisticated and as yet undiscovered algorithm to 
decipher that spatial information from the retinal input. 
Furthermore, in the absence of a global framework to reg- 
ister the information from each saccade in its proper place, 
vision as described by O’Regan would be indistinguishable 
from apperceptive agnosia, a visual integration failure. In 
other words, the condition of apperceptive agnosia is the 
absence of a visual function whose existence O’Regan ef- 
fectively denies. McLoughlin is right that the brain need 
not explicitly represent more than it requires at any partic- 
ular time, and it can make do with a sparse or abbreviated 
representation of the world. But he misses the paradigmatic 
point that the world we observe in experience is already that 
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sparse representation, the real world beyond experience 
being infinitely more complex than our experience of it. So 
the brain must explicitly encode exactly as much detail as 
we observe in experience, no less, and unbiased phenome- 
nological observation clearly reveals a spatially structured 
world. 

Fox complains that I refute direct perception on the 
grounds that no plausible mechanism has ever been iden- 
tified neurophysiologically that accounts for the external 
nature of perception. “Yet,” says Fox, “there is growing 
physiological evidence to the contrary,” and he cites neuro- 
physiological findings in the brain. But the kind of physical 
evidence required to support direct perception would have 
to be energy or information located outside the physical 
brain, out in external space where perception is supposed 
by direct realism to occur. Fox chides me that “Using the 
term ‘perceptual processing’ or ‘computation’ is a serious 
misrepresentation of direct perception.” He is quite cor- 
rect. But that is exactly what is wrong with Gibson’s theory 
of direct perception, and that is exactly why modern pro- 
ponents of Gibson’s theories usually take care to disclaim 
his most radical views. For if perception is not a computa- 
tion in the brain based on sensory input, then why does Fox 
cite evidence from the brain to explain that perception? Fox 
suggests “A more fruitful heuristic for understanding per- 
ception is a physiology that has evolved a sensitivity to 
meaningful environmental relational information or . . . ac- 
tion-oriented systems.” And how would one build an artifi- 
cial system with a “sensitivity to meaningful environmental 
relational information” that is not attained by way of input 
through sensory systems and internal representations? This 
“explanation” is every bit as mysterious as the property of 
consciousness it is supposed to explain. 

Lloyd disputes the phenomenological basis of the 
Gestalt Bubble model and insists that outside of focal at- 
tention he experiences only a very indefinite spatiality, 
which seems to him inconsistent with the continuously pre- 
sent three-dimensional model constructed in the Gestalt 
Bubble. Instead, he proposes that the natural supposition 
that our experience specifies a full 360-degree diorama 
arises from the “just-in-time” availability of spatial infor- 
mation with every attentional focus. But the availability of 
spatial information is not only “just in time,” but, more sig- 
nificantly, it is also “just in place,” that is, the spatial percept 
appears at the point in the global experience of three-di- 
mensional space that the percept is perceived to occupy in 
that space. Lloyd’s Gibsonian view also fails to account for 
dreams and hallucinations, where the world as an external 
memory is no longer available for data access, and yet a 
structured world is experienced nonetheless. There is no 
question that there is a loss of resolution in peripheral vi- 
sion — that too is easily confirmed phenomenologically. But 
if Lloyd’s experience of each individual saccade appears 
separate and disconnected from any global whole, like a se- 
ries of scenes on a television screen, then either he is suf- 
fering a form of apperceptive agnosia, or more likely, his 
theory of vision suffers from apperceptive agnosia, which in 
turn handicaps his phenomenological observations. This 
suspicion is supported by Lloyd’s own analysis of the di- 
mensions of conscious experience. The basic dimension, 
according to Lloyd, is temporal, and experience is an or- 
derly ensemble of phenomenal leaps and bounds along a 
time line. Spatiality emerges from trajectories encoded in 
proprioception, which orient each momentary percept to 
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those before and after. This is the consequence of design- 
ing a phenomenology based on one’s theory of perception, 
rather than a theory of perception based on one’s phenom- 
enology! 


R7. Neurophysiological issues 


McLoughlin argues that the fragmented architecture of 
the visual cortex into separate retinotopic maps requires a 
fragmented model of vision. But that is true only for a 
purely neurophysiological model that cares nothing about 
phenomenology, where the unity of visual experience is its 
most salient feature. But a neuroscience that explains 
everything about the brain except how it generates con- 
sciousness, is a neuroscience that explains nothing, because 
it is consciousness that makes the brain interesting in the 
first place. To declare from the outset that the unity of con- 
sciousness requires no explanation is to guarantee that no 
explanation will ever be found. 

Ross agrees that a simple neuron doctrine cannot ac- 
count adequately for size constancy in perception, but con- 
tends that more complex neurological models show 
promise. She then cites a number of neurophysiological 
models that account for some aspect or other of size con- 
stancy. But curiously, none of the models that Ross cites ac- 
counts for the one aspect of size constancy that is the cen- 
tral focus of the target article; that is, the fact that objects 
in space appear as solid volumetric objects embedded in a 
volumetric surrounding space, and that space has the pe- 
culiar property that its size scale shrinks progressively in 
nonlinear fashion with distance from the egocentric point. 
Both Gibson and the Gestaltists complained about the 
trend in psychophysics of breaking the complex phenome- 
non of visual experience into very simple visual tasks that 
are then recorded as keypress data points in psychophysi- 
cal studies. Neural network or other models are then de- 
vised to replicate those data points, and those models are 
then considered to be models of vision. Lost in the shuffle 
is the rich and complex volumetric spatially extended expe- 
rience of visual consciousness, which never finds its way 
into those models of vision. 

Grossberg is quite right when he says that the Gestalt 
Bubble model “makes no contact with neurophysiological 
and anatomical data about vision.” This means either that 
the model is wrong, or that neuroscience is in a state of se- 
rious crisis because it offers no hint of an explanation for the 
observed properties of conscious experience. If the latter 
should happen to be the case, as the target article suggests, 
then limiting our observations of our phenomenal experi- 
ence to that which is allowed by contemporary theories of 
neural representation will turn out to have been an exercise 
in futility. 

Duch also complains that I misrepresent the neuron 
doctrine by omitting discussion of dynamic recurrent 
neural networks. He must have missed my discussion of the 
dynamic recurrent neural network models of Grossberg, 
and their fundamental difficulties with modeling spatial ex- 
perience (sect. 3 of the target article). 

MacKay bolsters the evidence for Gestalt processes in 
the brain by considering the web of continuous electrical 
activity stretching from the spinal cord to the cerebrum. 
MacKay proposes that the “panexperientialist” view sug- 
gests that awareness is linked to something like an electri- 


cal field of this sort. Indeed, that is exactly why Kohler was 
so interested in electrical fields. My own preference is for 
a harmonic resonance theory (Lehar 2003) involving pat- 
terns of electrochemical standing waves in the neural sub- 
strate. Standing waves inherit all the properties of static 
electric fields, and add to them an extraordinarily rich 
repertoire of spatiotemporal behaviors that are very 
Gestalt-like in nature. This hypothesis also resolves the is- 
sue of integration raised by MacKay, because it is in the very 
nature of different resonances in a mechanism to couple to 
each other and thereby produce a single larger integrated 
resonance, of which the original resonances become higher 
harmonics (Lehar 2003b). 


R8. Various and sundry issues 


Laming raises the homunculus objection, that if there 
were picture-like processes active in perception, then there 
would have to be an internal viewer of those picture-like 
processes. I refuted this objection in the target article with 
the argument that information encoded in the brain needs 
to be available only to other internal processes rather than 
to a miniature copy of the whole brain. Laming rejects this 
explanation with the statement “The fact that Lehar has a 
mathematical model to replace the neurophysiological ob- 
servations does not alter this requirement.” But the re- 
quirement for an internal observer of any spatial data is it- 
self a paradigmatic assumption on Laming’s part. He has 
not shown that it is necessary in the first place, and it is at 
least equally likely prima facie that it is not. Furthermore, 
we know for a fact that our experience is expressed in the 
form of a spatial structure, regardless of whether that struc- 
ture requires an observer, and that experienced structure 
can be expressed in a perceptual model. There is no reason 
a model of perceptual experience should be invalid. 

Luccio takes issue with my characterization of Gestalt 
theory as a representationalist theory; he claims that it is 
neither representationalist nor antirepresentationalist but 
is merely “indifferentist” to the epistemological question. 
There have been different schools of Gestalt, not all of 
which have shared the same philosophy. But at least Koffka 
and Kohler, and therefore by implication presumably 
Wertheimer, were definitely representationalists. Koffka 
makes the most clear representationalist case with his dis- 
tinction between the “geographical environment” (the ob- 
jective external world) and the “behavioral environment” 
(the phenomenal world), and he clearly stated that the be- 
havioral environment is located inside the geographical 
body in the geographical environment (see Koffka 1935, 
p. 40, Fig. 2). Kéhler expressed his representationalist 
views most clearly in Kéhler 1971, p. 125. 

That is not to say that one can't be a Gestaltist and a di- 
rect realist. One can profess, like Gibson, that illusions are 
not illusory at all, and that perceived illusory surfaces have 
areal objective existence out in the physical world, although 
that existence cannot be verified by scientific means, and 
the function of the sense organs becomes highly ambigu- 
ous. In my view, the message of Gestalt has been represen- 
tationalist from the very beginning, with its focus on objects 
experienced vividly in phenomenal space that are known to 
have no objective existence. 

Markovieé is puzzled that I can claim at one point “the 
internal perceptual representation encodes properties of 
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the distal object rather than of the proximal stimulus” (sect. 
9, last para.) while at another I state “the direct realist view 
is incredible because it suggests that we can have experi- 
ence of objects out in the world directly, beyond the sen- 
sory surface, as if bypassing the chain of sensory process- 
ing” (sect. 2.1, first para.). Why, asks Markovié, would the 
thesis that distal objects are mapping onto the phenome- 
nological domain without neural intervention be incredible 
and mysterious, while the idea about the projection of in- 
ternal representation onto the external perceptual world 
not be incredible and mysterious? How is it possible that 
perception is partially indirect (representational), and par- 
tially direct (distally oriented)? 

Perception is entirely indirect; what we experience is in 
every sense inside our physical head. The “distal orienta- 
tion” of perception is seen in the form or dimensions in 
which perceptual information is expressed. The perceived 
world is expressed not in terms of the proximal image on 
the sensory surface, that is, a two-dimensional pattern of 
brightnesses, but in terms of actual three-dimensional ob- 
jects and surfaces in a world that we take to be reality. We 
do not see visual patterns and infer them to be a table, we 
experience a table, expressed in terms of volumes of per- 
ceived wood embedded in a volume of perceived space. But 
that world information does not enter experience directly 
in some magical mystical manner, but indirectly by the con- 
ventional route of sensory input, and that input is expanded 
out or reified in the brain to become the spatial percept that 
we experience. 

Randrup complains that my position is not really mate- 
rialist, because I say in the target article that “there remains 
a vivid subjective quality (or quale), to the experience of red” 
for example, which “is not in any way identical to any exter- 
nally observable physical variable” (sect. 3, para. 6) in the 
brain. This quoted passage however represents not my own 
view, but my summary characterization of Chalmers’ “hard 
problem” of consciousness, and why it is considered by some 
to imply a fundamental dualism. According to identity the- 
ory, the difference between subjective experience and its 
objective physical realization is a difference in viewpoint or 
perspective, rather than an ontological dualism, and that 
makes the Gestalt Bubble a materialist position. 

Randrup himself favors an idealist position, and goes on 
to conclude that the Gestalt Bubble model is most readily 
understood within the idealist world view, whereby the 
troubles of direct or indirect perception are significantly re- 
duced. It is true that the Gestalt Bubble model is useful 
even as a structural description of pure mind. But to deny 
the existence of an independent objective material world, 
of which that mind is a copy, strains my credulity beyond its 
elastic limit. 

Rosenthal & Visetti are generally supportive of the per- 
ceptual modeling approach proposed in the target article. 
However, they are puzzled about whether the proposed 
mechanism of emergence in the model is motivated pri- 
marily by the emergent properties of perception, or 
whether it is a physicalist model, whose spatial matrix and 
fieldlike interactions represent physical space and physical 
forces in the brain. 

In the first place, the model is explicitly defined as a 
model of experience, and the local elements in the model 
are defined as local perceptual experiences. The dynamic 
fieldlike forces are therefore defined as perceptual tenden- 
cies observed phenomenologically, for example, the ten- 
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dency for perceived surfaces to fill in like a milky soap bub- 
ble, and the tendency for corner or occlusion percepts to 
link up to produce globally coherent edges. Although the 
dynamics of these experiences are usually so fleeting as to 
be impossible to observe, it is the configuration of the end 
result, or final stable percept, that implicates an emergent 
spatial filling-in, because no other mechanism could plau- 
sibly produce that result. So the Gestalt Bubble model is 
not a physicalist model of the brain, but a mathematical 
model of experience, although it is committed to an emer- 
gent spatial computational strategy as offering the best ex- 
planation for spatial experience, and that in turn sets con- 
straints on the corresponding neurophysiological mechanism 
in the brain. 

Schirillo proposes to extend the Gestalt Bubble model 
by adding the perception of illumination to that of spatial 
structure. Schirillo is an astute phenomenological observer; 
the perception of brightness, lightness, and illumination are 
indeed intimately coupled to the perception of visual struc- 
ture. I have explored the interaction of spatial perception 
to the perception of illumination in Chapter 5 of my book 
(Lehar 2003b). 

Tse is generally supportive of the Gestalt Bubble model, 
and offers a more general analysis of the why the visual sys- 
tem operates as it does. Tse identifies two general princi- 
ples by which the visual system attempts to correct errors. 
One involves completing missing information on the basis 
of knowledge about what most likely exists in the scene, that 
is, perceptual “filling-in”; and the other involves exploiting 
the physical stability of the environment as a reference 
frame with respect to which the eyes and body can move. 
An interesting aspect of this view is that the visual system 
implicitly understands its own limitations, and attempts 
perceptual filling-in only when it “knows” that it has failed 
to detect something that it believes must be present. 

I take issue, however, with Tse’s contention that in 
amodal completion there is no perceptual filling-in. It is 
true that there is no modal filling-in of explicit surfaces, as 
in the Kanizsa figure, which is what Tse probably intended. 
But there is filling-in nonetheless, although ofa nonsensory, 
amodal manner. When we see an occluded object, like a 
horizontal branch, part of which is occluded by a nearer ver- 
tical tree trunk, we can reach back behind the tree trunk 
and grab exactly that point in space that we “know” to be 
occupied by the occluded branch, based exclusively on the 
configuration of its visible portions. Although it is a seman- 
tic issue whether such experience is really “seeing” at all, 
there is no question that a three-dimensional volumetric ex- 
perience is involved, and that experience is produced by fill- 
ing-in processes very much like those seen in the modal sur- 
faces of the Gestalt illusions. 

Wright supports the representational stance in the tar- 
get article, and provides further arguments to defeat the al- 
ternative direct realist view. Wright objects, however, to the 
use of the term “subjective” when applied to the sensory 
field, saying the sensory representation of colored volumes 
embedded in perceived space is thoroughly nonconceptual, 
and therefore no kind of subjective judgment is involved at 
that low level of experience. If television is an apt analogy 
for the televisual function of vision, Wright suggests I ac- 
knowledge the nonconceptual nature of the pattern of 
glowing phosphor dots on the television screen. 

The semantic distinction that Wright draws between 
nonconceptual sensory processes and subjective judgment 
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based on that sensory data may serve him well for his own 
purposes, but it is at odds with a prominent theme in 
Gestalt theory: that there is no difference in essential prin- 
ciples between the lower-level functions of sensation and 
perception, and the higher-level functions of recognition 
and cognition, except for a difference in complexity (cf. 
Lehar 2003b, Ch. 6). The higher-level recognition ofa table 
as a whole is not different in principle from the recognition 
of its edges and surfaces as edges and surfaces. In the tele- 
vision analogy, the individual pixels of a photosensor array 
can be seen as very simple “feature detectors” tuned to re- 
spond to their feature, the brightness and color of light from 
a narrow angle of the visual field. Similarly, the local spatial 
fields proposed in the Gestalt Bubble model can be seen as 
three-dimensional surface-element, edge-element, and 
comer-element “feature detectors” that, in cooperation and 
competition with their neighbors, make a collective “sub- 
jective judgment” about the presence or absence of edge or 
comer features in particular parts of space. What is missing 
in the Gestalt Bubble model is the strict input/output func- 
tion normally ascribed to feature detectors, because the 
“output,” or final state of a particular “detector,” depends 
not only on the input from the retina, or only on local in- 
teractions in perceived space, but on the total configuration 
of all of the other local elements across the whole of phe- 
nomenal space simultaneously. Even the highest-level 
global recognition has an influence on the state of the low- 
est-level edges and surface brightnesses of a scene, as seen 
in the subjective reversals of Figure 11D of the target arti- 
cle. That perceived comers and surfaces are observed to 
change their configuration with the perceptual shift, clearly 
indicates the “subjective” nature of these low-level compo- 
nents of experience, which are not strictly invariant with the 
input, as Wright suggests. 
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